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1. Introduction 


This simulation study examines the measurement precision, item exposure rates, and the depth 
of the MAP® Growth™ item pools under various grade-level restrictions. Unlike most summative 
assessments, MAP Growth allows examinees to see items from any grade level, regardless of 
the examinee’s actual grade level. It does not limit the test to items designated for the 
examinee’s actual grade. A benefit of this flexibility is that the test may adapt to the examinee’s 
ability level regardless of grade, which can avoid examinee frustration and emotional trauma 
(Thurlow, Elliott, & Ysseldyke, 1999). It can also substantially improve the measurement of 
students who are performing significantly below or above the grade level. In addition, this 
approach is consistent with measuring growth rather than just status for accountability 
purposes. As long as test scores and the vertically articulated standards are expressed on the 
same scale, each student can be compared to the same “on-grade” standards regardless 
whether or not they received off-grade items in their individual test (Way et al., 2010). The 
downside is that the examinee may see items that are not aligned to the curriculum for a 
particular grade. A developmental scale that is not restricted to grade-level content enables a 
teacher to identify where students are in their learning, and to modify instruction accordingly. 
Low performing students may need remedial content before grade-level content. A high 
performing student may need to be challenged with above-grade content during instruction. 


On the other hand, teachers are accountable for teaching grade-level content. Grade level 
serves as the organizing basis for curriculum and instruction, therefore a test focused on grade- 
level content arguably provides a better measure of on-grade learning. Ifa test is used to drive 
changes in instructional practices, on-grade assessment is more appropriate than out-of-level 
assessment (Thurlow, Elliott, & Ysseldyke, 1999). However, researchers found that student 
performance on grade-level assessment is likely to be higher than their performance on out-of- 
level assessments due to overestimation provided by high proportions of chance level scores 
(Doscher & Bruno, 1981; Wick, 1983). 

The purposes of this research study are as follows: 


1. To explore different possible test designs using on-grade items only with the current 
MAP Growth item pools 
2. To provide recommendations to Content Solutions for developing grade-level items 


Specifically, this study evaluates the depth of MAP Growth on-grade item pools along the 
student ability spectrum based on empirical Rasch Unit (RIT) score distributions. The study 
manipulated two factors: the type of item pool and type of simulee. Different item pools were 
created for the simulations with the following restrictions to explore the feasibility of creating on- 
grade MAP Growth assessments. Simulee groups were based on on-grade and off-grade 
student ability. 


e On-grade item pools that were limited to on-grade items only. For example, the Grade 4 
item pool was created for simulees assigned to Grade 4. 

e +1 grade item pools that contained items for the target grade plus items one grade below 
and one grade above the target grade. For example, a Grade 4 simulee would encounter 
items appropriate for Grades 3, 4, and 5. 

e All-grade item pools that are used in the current MAP Growth 2-5, 6+, and 2—12 tests 
and have no grade-level restrictions. A simulee could see items from all possible grade 
levels. 
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1.1. Background 

Four major components of a computerized adaptive test (CAT) are the item pool, item selection 
method, ability estimation method, and stopping rule (Reckase, 1989). Of these components, 
item pool quality is the foundation for a CAT to achieve desirable psychometric properties 
(Veldkamp & van der Linden, 2000; Xing & Hambleton, 2004; van der Linden, Ariel, & 
Veldkamp, 2006; Reckase, 2010). A high-quality item pool should contain an adequate number 
of items that support efficient and informative testing across important ability levels (Wise, 
1997). The item pool must also “contain high-quality items for many different levels of 
proficiency” for a test that claims to measure students at different ability levels (Flaugher, 2000, 
p. 38). If the item pool is shallow in some ability levels, the CAT will be less efficient in terms of 
ability estimation (Wise, 1997). 


Wise (1997) evaluated CAT item pools regarding pool size, adequacy of the test information 
provided by the pool, and balance of content domains and item difficulties. He provided 
recommendations regarding targeted expansion of the item pools and adopting more efficient 
testing methods (e.g., the AMT" procedure). More relevant to the current study, Wei and Lin 
(2015) investigated the use of out-of-level items versus grade-level items in CAT through 
simulations. In their study, items from adjacent grades were treated as out-of-level items and 
were pooled together with in-level items. Their findings indicated that administering out-of-level 
items improved measurement accuracy and test efficiency for low- and high-performing 
students but did not impact students in the middle of the ability spectrum. 


1 Adaptive mastery testing (Weiss & Kingsbury, 1984). 
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1.2. Data 
The following item pools and data were used in this simulation study: 


e MAP Growth Mathematics, Reading, and Language Usage item pools for Grades 3-8 
aligned to the Common Core State Standards (CCSS; National Governors Association 
Center for Best Practices & Council of Chief State School Officers [CCSSO], 2010)? 
based on the three item pool restrictions (i.e., on-grade, +1 grade, and all-grade) 

e Study sample: test event data extracted from the Growth Research Database (GRD) for 
the 2016-2017 school year 


As shown in Table 1.1, the simulations included 21 on-grade item pools, 21 +1 grade item 
pools, and five all-grade item pools. On-grade and all-grade item pools were obtained directly 
from item pool data. For the +1 grade item pools, items from three grades were combined and 
the number of items was obtained by counting unique item IDs across the three grade levels. 
Test event data was obtained from CCSS MAP Growth tests administered in 2016—2017. Table 
1.2 summarizes student RIT distributions from those test events. 


Table 1.1. Simulation Study Item Pools 


On-Grade +1 Grade All-Grade 

Content Area Grade Item Pool #ltems | Item Pool #ltems | Item Pool #ltems 
2 Math 2 689 Math 1-3 1,926 
3 Math 3 928 Math 2-4 2,543 

4 Math 4 1,020 Math 3-5 2,598 Mes oe 
Mathematics 5 Math 5 822 Math 4-6 2,815 
6 Math 6 1,166 Math 5-7 2,742 

7 Math 7 945 Math 6-8 2,716 Math 6-8 6,657 
8 Math 8 718 Math 7-9 2,824 
2 Read 2 584 Read 1-3 1,324 
3 Read 3 1,072 Read 2-4 2,012 

4 Read 4 1,216 Read 3-5 2,271 Rega Set 
Reading 5 Read 5 969 Read 4-6 2,441 
6 Read 6 1,036 Read 5-7 2,106 

7 Read 7 697 Read 6-8 1,893 Read 6-8 6,472 
8 Read 8 710 Read 7-9 1,382 
2 Lang 2 445 Lang 1-3 1,621 
3 Lang 3 922 Lang 2-4 1,771 
4 Lang 4 1,004 Lang 3-5 1,910 

eon 5 Lang 5 681 | Lang4-6 1,730 | Lang2-12 5,536 
6 Lang 6 512 Lang 5-7 1,284 
7 Lang 7 509 Lang 6-8 1,005 
8 Lang 8 366 Lang 7-9 861 


2 © Copyright 2010 National Governors Association Center for Best Practices and Council of Chief State 
School Officers. All rights reserved. 
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Table 1.2. Student RIT Distributions from 2016-2017 CCSS Test Event Data 


RIT Descriptive Statistics 
Content Area Grade | #Students Mean SD Median Skewness’_ Kurtosis 

2 1,386,531 183.00 15.97 183.50 -0.21 0.41 

3 1,464,118 194.00 14.86 194.61 -0.33 0.72 

4 1,454,385 | 205.00 15.56 205.34 -0.31 0.96 

Mathematics 5 1,457,360 | 214.00 16.95 214.03 -0.26 0.75 
6 1,414,750 | 217.00 16.99 218.14 -0.35 0.73 

7 1,356,673 | 223.00 18.39 224.28 -0.33 0.55 

8 1,301,542 | 229.00 19.33 229.42 -0.32 0.53 

2 1,351,809 180.33 17.80 181.33 -0.12 -0.59 

3 1,445,055 19165 17.44 193.68 -0.41 -0.24 

4 1,440,187 | 200.66 16.91 202.82 -0.63 0.39 

Reading 5 1,440,237 | 207.52 16.59 209.43 -0.73 0.82 
6 1,374,256 | 212.27 16.34 214.28 -0.83 1.21 

7 1,329,350 | 216.39 16.41 218.38 -0.87 1.40 

8 1,288,344 | 220.21 16.33 222.15 -0.91 1.64 

2 239,959 180.49 16.93 180.88 0.08 -0.89 

3 378,455 192.00 16.14 194.04 -0.38 -0.42 

4 411,293 | 200.52 15.41 202.62 -0.64 0.27 

a 5 410,999 | 206.65 14.91 208.60 -0.79 0.85 
6 426,644 | 211.13 14.85 213.02 -0.84 1.17 

7 405,557 | 21489 14.79 216.68 -0.86 1.41 

8 393,550 | 218.37 14.78 220.02 -0.85 1.60 
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2. Method 


CAT simulations were performed using the SAS sandbox tool (Wang, 2016). To ensure that the 


simulations accurately reflected the operational MAP Growth tests, the test length, item 
selection by content balancing, and standard error of measurement (SEM) criteria for the 
simulations were the same as those used in the operational MAP Growth tests. Owen’s 


Bayesian method (Owen, 1975) was used for momentary person ability calculations, and the 
maximum likelihood estimation (MLE) method was used final ability estimation. All item difficulty 


values and person ability scores were on the RIT scale, which has a mean of 200 anda 


standard deviation (SD) of 10 for each content area. Table 2.1 presents the test specifications 


followed in the simulations. While the number of administrations per year and longitudinal 


constraint are used in operational test validation, they are not considered at this point in this 


simulation study. 


Table 2.1. Test Specifications 


Test Grades | Length Instructional Area 

Geometry 

MAP Growth Measurement and Data 

. 2-5 50 ; 
Mathematics 2-5 Number and Operations 

Operations and Algebraic Thinking 
Geometry 

MAP Growth Operations and Algebraic Thinking 

: 6-8 50 
Mathematics 6+ Statistics and Probability 

The Real and Complex Number Systems 
Informational Text: Key Ideas and Details 
Informational Text: Language, Craft, and Structure 

MAP Growth ; ; . 

Reading 2-5 2-5 40 Literary Text: Key Ideas and Details 
Literary Text: Language, Craft, and Structure 
Vocabulary: Acquisition and Use 
Informational Text: Key Ideas and Details 
Informational Text: Language, Craft, and Structure 

MAP Growth F . : 

Reading 6+ 6-8 40 Literary Text: Key Ideas and Details 
Literary Text: Language, Craft, and Structure 
Vocabulary: Acquisition and Use 
Language: Understand, Edit for Grammar, Usage 

ni Grow 2-8 50 Language: Understand, Edit for Mechanics 


Language 2-12 


Writing: Write, Revise Texts for Purpose and Audience 


Note. Length is the number of items on the assessment. 


2.1. Simulation Conditions 
The simulation study design implemented the following two factors that were crossed to yield six 
conditions (e.g., on-grade item pool with on-grade student ability; on-grade item pool with off- 
grade student ability; +1 grade item pool with on-grade student ability; etc.). In each condition, 
500 simulees were assigned to each true RIT point. 
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1. Item pool (on-grade, +1 grade, and all-grade) 
2. Student ability level (students with on-grade ability and students with off-grade ability) 


2.1.1. On-Grade Student Ability 


Student's true RITs were drawn from empirical distributions summarized from the 2016-2017 
CCSS MAP Growth test scores in Table 1.2. Seven RIT points (R1 — R7) were obtained for 
each grade and content area based on the mean and SD of the student RIT distribution. Table 
2.2 presents the true RIT points for each grade and content area. R4, the middle point, was the 
mean. RIT points ranged from two SDs below the mean to two SDs above the mean (i.e., -2SD 
to 2SD). Specifically, the RIT points below the mean were -2SD (R1), -1.5SD (R2), and -1SD 


(R3), and the RIT points above the mean were 1SD (R5), 1.5SD (R6), and 2SD (R7). For 


simulation conditions with on-grade simulees, a total of 3,500 test events (500 x 7 = 3,500) were 
simulated for each grade and content area. 


Table 2.2. True RITs for On-Grade Student Ability 


True RIT 

Content Area | Grade R1 R2 R3 R4 (Mean) R5 R6 R7 
2 151 159 ~+~«167 183 199 207 215 

3 164. 172 179 194 209 «-216~=—S( 224 

4 174. 181 +189 205 220 228 236 

Mathematics 5 180 188 197 214 230 239 247 
6 183 192 200 217 234 «-243-—Ss«2851 

7 187 196 205 223 242 251 +260 

8 190 200 209 229 248 258 267 

2 145 154 163 180 198 207 216 

3 157 165. ~— 174 192 209 +218 ~=— 227 

4 167 175 184 201 218 226 234 

Reading 5 174 183 191 208 224 «232—~Sés«2 
6 180 188 196 212 229 «-237-—Ss 245 

7 184 192 200 216 233 «241 ~~ 249 

8 188 196 204 220 237-245-253 

2 147. 155.—Ss«164 180 197 206 214 

3 160 168 176 192 208 216 224 

4 170 177 185 201 216 224 231 

oe 5 177, 184.—Ss« 192 207 222 229 236 
6 181 189 196 211 226 ©233~Cts«21 

7 185 193 200 215 230 237 244 

8 189 196 204 218 233 «241 ~—Ss248 


2.1.2. Off-Grade Student Ability 
For the off-grade student ability values, six true RIT points (R1 — R6) were obtained from 


empirical analysis of extant data for each grade and content area. True RITs for off-grade 
students were defined by 1 grade down and 1 grade up. Table 2.3 presents the true RIT points 
for each grade and content area. R1, R2, and R3 represent the lower grade RITs, and R4, R5, 


and R6 represent the higher grade RITs. There were no on-grade student abilities in this 


condition. For simulation conditions with off-grade simulees, a total of 3,000 test events (500 x 6 


= 3,000) were simulated for each grade and content area. 
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Table 2.3. True RITs for Off-Grade Student Ability 


1 Grade Down 1 Grade Up 
Test Grade R1 R2 R3 R4 R5 R6 
2 133 138 142 | 216 220 #224 
3 151 159 162 | 228 232 236 
4 164 168 172 | 239 243 247 
Mathematics 5 174 176 179 | 248 249 ~# 251 
6 180 181 182 | 254 257 260 
7 183 184 185 | 262 264 267 
8 187 189 190 | 268 269 270 
2 134 138 142 | 218 223 227 
3 145 150 154 | 229 232 234 
4 157 +161 +165 | 236 238 241 
Reading 5 167 170 172 | 242 244 245 
6 174 176 179 | 246 248 249 
7 180 182 183 | 250 252 253 
8 184 186 187 | 254 255 256 
2 134 138 142 | 216 220 224 
3 155 157 159 | 226 228 231 
4 160 165 169 | 232 234 236 
ca 5 170 173 176 | 237 239 241 
6 177. 179 180 | 242 243 244 
7 181 183 184 | 245 247 248 
8 185 186 188 | 249 250 251 


2.2. Statistical Analyses 
Measurement precision was evaluated for each condition with the following statistics: 


e Test information. The information provided by an observed test. A large amount of 
information means a student with the same ability as the true RIT can be estimated with 
precision. Relative efficiency (RE) was computed to show the degree of additional test 
information that an all-grade item pool provided over an on-grade item pool and a +1 
grade item pool. RE > 1 means the all-grade item pool was more efficient, whereas RE < 
1 means the alternative item pool was more efficient. Relative efficiency provides an 
indication of how many items are needed to achieve comparable amount of information. 
For example, a relative efficiency of 1.5 indicates that a 50% increase in the number of 
items on a test produced from the on-grade pool is needed to achieve the same 
information as test produced from the all-grade pool. 


e SEM. The SEM was computed for each simulee as the inverse of the square root of test 
information. For MAP Growth tests, SEM varies due to different test lengths. An SEM 
less than 3.0 was observed in typical mathematics and language usage tests (50 items), 
and an SEM less than 3.4 was observed in typical reading tests (40 items). 
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e Bias and RMSE. Bias and root mean square error (RMSE) summarize the difference 
between true RIT and the final estimated RIT score. The equations to compute these 
statistics are as follows: 


Bias = N~* iL, (6, - 4;) (1) 


RMSE = |w-1 yi,(6,— 6))° @) 


where 6; is the true RIT, and 6; is the estimated RIT score. 
The item pool was also evaluated for each condition with the following variables: 


e /tem exposure rate. |tem exposure rate based on item grade and content strand was 
calculated as the percentage of total items administered in each condition. 

e Pool utilization. The number and percent of items administered were calculated and 
provide an indication of the degree to which the item pool was used in each condition. 


Finally, content balancing (i.e., content coverage) was examined against the expectation that 
items are distributed approximately equally across instructional areas within each test. Content 
coverage was examined by calculating the percentage of an instructional area in each test that 
had balanced, slightly unbalanced, or extremely unbalanced item counts, with consideration of 
power selection (i.e., items are selected at the beginning of the test without regard to 
instructional area), as shown in Table 2.4. 


Table 2.4. Flag Criteria for Content Balancing 


Total #ltems in an Slightly Extremely 
Content Area #Iltems | Instructional Area Balanced | Unbalanced | Unbalanced 
Mathematics 50 4 seneiy | |) 
Reading 40 5 5<n<13 es be ars 
Language Usage 50 3 13 <n<21 S i . pe jeans 
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3. Results 


3.1. Measurement Precision 

Appendix A presents the SEM and relative efficiency results, and Appendix B presents the bias 
and RMSE results. The results varied by content area, so separate summaries are provided 
below. 


3.1.1. SEM and Relative Efficiency 

3.1.1.1. Mathematics 

As shown in Table A.1, SEMs for on-grade simulees are consistently larger when using an on- 
grade mathematics item pool. For on-grade simulees exposed to an on-grade item pool, the 
range of SEM was large (2.96 — 5.20). Many values were larger than 3, and the largest values 
occurred at the low end of the scale. The average SEM for the on-grade pool is 3.16, which is 
above the desirable value for precise measurement (i.e., SEM < 3). In the worst case (Grade 8, 
RIT = 190), the on-grade test would need to be 2.8 times longer than a test from the all-grade 
pool to achieve the same amount of information. 


SEMs for the +1-grade and all-grade item pools are lower than those for the on-grade pool as 
evidenced by the average SEM values below three. Moreover, the +1-grade and all-grade pools 
had similar SEMs. Relative efficiency values also show that the on-grade pool is consistently 
less efficient than the all-grade pool when given to on-grade students. The +1-grade pool tends 
to be as efficient as the all-grade pool for on-grade students. 


As grade level increases, the relative efficiency of the on-grade pool decreases at the low end of 
the scale. It decreases in a similar way for the +1 grade pool but to a lesser extent. This pattern 
suggests that the on-grade pool lacks items at the low end of the scale, resulting in less precise 
measurement there. 


SEMs were uniformly larger for off-grade simulees than they were for on-grade simulees, 
regardless of the type of item pool, but the increase was largest for the on-grade pool (see the 
right panel of Table A.1). In fact, the SEM ranged from 2.98 to 5.8 for the on-grade pool, but it 
only ranged from 2.95 to 3.39 for the all-grade pool. For off-grade simulees, the all-grade pool 
was 1.01 to 3.42 times more efficient than the on-grade pool for off-grade simulees, and it was 1 
to 1.37 times more efficient than the +1-grade pool. In almost all cases, the all-grade pool had 
SEMs less than 3. The opposite was true for the on-grade and +1-grade pools, where most 
SEMs were larger than 3. 


3.1.1.2. Reading 

As shown in Table A.2, SEMs for reading were generally higher than they were in mathematics 
for both on-grade and off-grade simulees. The all-grade item pool had the lowest SEMs, 
followed by the +1-grade pool and the on-grade pool, but all SEMs were larger than 3. The all- 
grade pool was the only pool that had an average SEM of 3.4 (the operational stopping rule for 
SEM), and it had the smallest range of SEMs (3.34 to 3.87). The +1-grade pool also had a 
slightly higher average SEM (3.41), and wider range of values (3.32 to 4.14). The on-grade pool 
had the largest average SEM with a value of 3.59. It also had the widest range of values (3.33 
to 5.06). For each pool, values larger than 3.4 typically occurred at extreme ends of the scale, 
particularly for the on-grade pool. It was usually the low end of the scale that resulted in SEMs 
larger than 3.4 for the on-grade pool. 
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Relative efficiencies also show that the +1-grade pool produced comparable information to the 
all-grade pool for on-grade simulees. In some cases, the +1 grade pool (e.g., Grade 5, RIT = 
183) and the on-grade pool was more efficient than the all-grade pool (e.g., Grade 6, RIT = 
212). However, the all-grade item pool was generally more efficient than the +1-grade pool and 
on-grade pool with efficiencies as high as 1.27 and 1.92, respectively, when compared to the 
all-grade pool for on-grade simulees. These values indicate that some +1-grade tests would 
have to be about 1.3 times longer than an all-grade test. Similarly, some on-grade tests would 
have to be almost twice as long as an all-grade test to yield the same amount of information. 


For off-grade simulees (right panel of Table A.2), SEMs were consistently larger than they were 
for on-grade simulees regardless of the item pool. Across all grades and RIT scores, the 
average SEM was 3.50, 3.87, and 4.64 for the all-grade, +1-grade, and on-grade item pools, 
respectively. The range of SEMs was lowest for the all-grade pool (3.33 — 5.52), and the +1- 
grade pool had a narrower range (3.34 — 6.00) than the on-grade pool (3.44 — 6.67). Relative 
efficiencies followed a similar pattern. The all-grade pool produced the most information for off- 
grade simulees, the +1-grade pool resulted in slightly less information, and the on-grade pool 
produced substantially less information than the all-grade pool. On average, a +1-grade test 
would need to be 1.2 times longer than an all-grade test, and an on-grade test would need to be 
1.66 times longer than an all-grade test. 


3.1.1.3. Language Usage 

As shown in Table A.3, SEMs for language usage pools were lower than they were for reading., 
All pools performed well for on-grade simulees, but the lowest SEMs occurred for the all-grade 
item pool. In a few cases, the on-grade (Grade 5, RIT = 184) and +1-grade (Grade 6, RIT = 221) 
pools were more efficient than the all-grade pool. The all-grade item pool had an average SEM 
less than 3 with a range of 2.96 to 3.65 points. The mean and range were larger for the +1- 
grade and even larger for the on-grade pool than they were for the all-grade pool. In terms of 
information, the +1-grade pool was nearly as efficient as the all-grade pool for on-grade 
simulees. Like mathematics, language usage showed a trend of increasing SEMs at the low end 
of the scale as the grade level increased, particularly for the on-grade pool. 


For off-grade simulees (right panel of Table A.3), SEMs were generally larger for all item pools 
than they were for on-grade students. The pattern of performance was similar to mathematics 
and reading. The all-grade item pool had lower SEMs across all grades and RIT levels (mean 
SEM = 3.14). The +1-grade pool had the next lowest SEMs (mean SEM = 3.29), and the on- 
grade pool consistently yielded the largest SEMs (mean SEM = 3.87). The +1-grade pool was 
almost as efficient as the all-grade pool with an average relative efficiency of 1.09. The on-grade 
pool had an average relative efficiency of 1.48 with the all-grade pool, indicating that an on- 
grade test would need to be almost 50% longer to yield the same information as an all-grade 
test. 


3.1.2. Bias and RMSE 

3.1.2.1. Mathematics 

As shown in Table B.1, biases for on-grade simulees are consistently larger when using an on- 
grade item pool (ranging from -0.4 to 1). The RMSEs for the on-grade pool are also relatively 
large with an average value of 2.53. Biases for the +1-grade and all-grade item pools are lower 
than those for the on-grade pool, ranging from -0.33 to 0.36 and -0.31 to 0.24, respectively. The 
average RMSE also dropped to 2.39 for the +1-grade item pool and to 2.36 for the all-grade 
pool. Similarly, for off-grade simulees (right panel of Table B.1), biases for the on-grade item 
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pool are larger (ranging from -0.41 to 1.23). The RMSEs for the on-grade pool are also larger 
with an average value of 2.88. Biases for the +1-grade and all-grade item pools ranged from - 
0.3 to 0.51 and -0.31 to 0.18, respectively. The average RMSE also decreased to 2.5 for the +1- 
grade item pool and 2.4 for the all-grade pool. 


3.1.2.2. Reading 

As shown in Table B.2, estimation accuracy for reading was generally not as good as 
mathematics for both on-grade and off-grade simulees. Biases for on-grade simulees are 
consistently larger when using an on-grade item pool (ranging from -0.92 to 0.62). The RMSEs 
for the on-grade pool are also relatively large with an average value of 2.89. Biases for the +1- 
grade and all-grade item pools are lower than those for the on-grade pool, ranging from -0.4 to 
0.44 and -0.33 to 0.37, respectively. The average RMSE also dropped to 2.76 for the +1-grade 
item pool and 2.71 for the all-grade pool. Similarly, for off-grade simulees (right panel of Table 
B.2), biases for the on-grade item pool are larger, ranging from -1.94 to 1.89. The RMSEs for 
the on-grade pool are also larger with an average value of 3.81. Biases for the +1-grade and all- 
grade item pools ranged from -0.81 to 1.26 and -0.39 to 1.19, respectively. The average RMSE 
also decreased to 3.12 for the +1-grade item pool and 2.8 for the all-grade pool. 


3.1.2.3. Language Usage 

As shown in Table B.3, estimation accuracy for language usage tests were better than reading. 
For on-grade simulees, biases are larger when using an on-grade item pool (ranging from -0.3 
to 0.74). The RMSEs for the on-grade pool are also relatively large with an average value of 
2.53. Biases for the +1-grade and all-grade item pools are lower than those for the on-grade 
pool, ranging from -0.4 to 0.35 and -0.25 to 0.27, respectively. The average RMSE also dropped 
to 2.42 for the +1-grade item pool and 2.39 for the all-grade pool. Similarly, for off-grade 
simulees (right panel of Table B.3), biases for the on-grade item pool are larger, ranging from - 
0.96 to 2.66. The RMSEs for the on-grade pool are also larger with an average value of 3.15. 
Biases for the +1-grade and all-grade item pools ranged from -0.27 to 1.96 and -0.25 to 1.43, 
respectively. The average RMSE also decreased to 2.66 for the +1-grade item pool and 2.51 for 
the all-grade pool. 


3.2. Item Exposure Rates 

Appendix C presents the item exposure rates by item pool and grade®*. For on-grade simulees 
(Tables C.1 — C.3), tests using the all-grade item pool had items spread out across item grades. 
In most conditions with on-grade simulees, the modal grade of the items administered to 
simulees was one to two levels away from the simulee’s actual grade, while in some conditions, 
the modal grade went up or down four levels. For example, in mathematics Grade 7, the modal 
grade went to Grade 3, Grade 4, Grade 6, and Grade 9 for various RIT levels. In reading Grade 
2, the modal grade went through Grades 2-6. In language usage Grade 8 tests, the modal 
grade went to Grade 4, Grade 6, Grades 9-10, and Grades 11-12. 


For tests with off-grade simulees (Tables C.4 — C.6), the modal grade was less spread out and 
mostly focused in two grades that could be as much as eight grade levels apart. For example, in 
mathematics Grade 6, the modal grade went down to Grade 3 when testing students with lower 
Grade 5 ability (RIT 180—182) and then went up to Grade 9 when testing students with higher 
Grade 7 ability (RIT 254—260). In reading Grade 2, the modal grade stayed in Grade 2 when 
testing students with lower Grade 1 ability and then went up to Grade 6 when testing students 


3 Item exposure rates by content strand are available in separate files. 
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with higher Grade 3 ability. In language usage Grade 3, the modal grade went down to Grade 1 
when testing students with lower Grade 2 ability and then went up to Grades 9-10 when testing 
students with higher Grade 4 ability. 


For simulations with on-grade simulees, tests using the +1-grade item pool mainly borrowed 
items from the lower or upper grade to support students with relatively lower or higher ability. 
For example, in reading Grade 2, more than 50% items were from Grade 1 to support students 
with lower Grade 2 ability, and more than 80% items were from Grade 3 to support students 
with higher Grade 2 ability. This pattern is more prominent in simulations with off-grade 
simulees. Tests using the +1-grade item pool borrowed a larger number of items from the lower 
and upper grades than they did on-grade to support students with lower or upper grade ability. 
For example, in reading Grade 5, more than 98% items were from Grade 4 when testing 
students with lower Grade 4 ability, and more than 92% items were from Grade 6 when testing 
students with higher Grade 6 ability. However, it is less obvious in mathematics tests in Grade 
2, Grade 4, Grade 5, and Grade 6, and reading and language usage tests in Grade 8 for 
simulations with both on-grade and off-grade simulees. 


3.3. Pool Utilization 

Appendix D presents the pool utilization results. In mathematics tests, the degree of item use is 
higher with the on-grade item pool (mean=53.7%) than for the +1-grade item pool 
(mean=46.3%) or all-grade item pool (mean=39.5%). For mathematics, 80% of items were from 
the on-grade item pool in 11 conditions. Reading results follow a similar pattern, with an 
average of 48.4% using the on-grade item pool versus 43.3% with the +1-grade item pool and 
37.1% with the all-grade item pool. For reading, 80% of items were from the on-grade item pool 
in seven conditions. Language usage tests have the highest item use rate, with an average of 
60.8% using the on-grade item pool versus 53.9% with the +1-grade item pool and 42.3% with 
the all-grade item pool. For language usage, 80% of items were from the on-grade item pool in 
13 conditions. 


Compared to simulations with on-grade simulees, item use rates are relatively lower in 
simulations with off-grade simulees. As shown in Tables D.4 — D.6, on average, mathematics 
tests have similar rates with different types of item pools (23.6% with the on-grade item pool, 
21.1% with the +1 grade item pool, and 25.1% with the all-grade item pool). Similarly, reading 
tests have similar use rates across item pools (13.1% with the on-grade item pool, 12.1% with 
the +1 grade item pool, and 15.7% with the all-grade item pool). Likewise, language usage tests 
have 23.9% with the on-grade item pool, 21.6% with the +1 grade item pool, and 25.2% with the 
all-grade item pool. 


3.4. Content Balancing 

Appendix E presents the content balancing results (i.e., how well the items were distributed 
within each test across instructional areas) by percentage of each instructional area that has 
balanced, slightly unbalanced, and extremely unbalanced item counts. 

For simulations with on-grade simulees, most instructional areas (> 96%) are balanced in 
mathematics tests with different item pools. Grade 5, Grade 6, and Grade 8 show unbalanced 
results. As shown in Figure E.1, Grade 5 items are extremely unbalanced along the instructional 
areas. The pool is shallow in “Operations and Algebraic Thinking” and “Geometry”. For the 
Grade 6 item pool shown in Figure E.2, the reason for unbalanced instructional areas may be 
that most items are associated with “The Real and Complex Number Systems.” As shown in 
Figure E.3, the Grade 8 item pool is shallow in “The Real and Complex Number Systems” and 
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“Statistics and Probability.” For reading, most instructional areas (> 97%) are balanced in tests 
with different item pools. Grades 6—8 show unbalanced results at the low end, especially with 
the on-grade and +1-grade item pools. As shown in Figures E.3 — E.6, except for “Vocabulary,” 
other instructional areas are relatively weak in the RIT range of 180-200. Content balancing 
were greatly improved with all-grade item pool. 


For simulations with off-grade simulees, most instructional areas (> 96%) are balanced in 
mathematics tests with different item pools. However, the Grade 5 item pool is unbalanced 
when administered to students with higher Grade 6 ability, and the Grade 6 item pool is 
unbalanced when administered to students with lower Grade 5 ability. For reading, on average 
(assuming students with off-grade ability take the grade-level reading tests), using the on-grade 
item pool caused content balancing to drop from 99.8% to 82.3%. Unbalanced instructional 
areas occurred when administering the following item pools: 


e Grade 2 item pool to students with lower Grade 1 ability or students with higher Grade 3 
ability 

e Grade 4 item pool to students with lower Grade 3 ability 

e Grade 6 item pool to students with higher Grade 5 ability 

e Grade 7 item pool to students with lower Grade 6 ability 

e Grade- 8 item pool to students with lower Grade 7 ability 


Language usage tests achieved the perfect results for simulations with both on- and off-grade 
simulees, and all three instructional areas were balanced in all test events. 
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4. Discussion 


The results in this study indicate that by using the on-grade or +1-grade item pool instead of the 
all-grade item pool, measurement precision, item exposure rate, pool utilization, and content 
balancing are diminished. Specifically, tests using the on-grade item pool may not be able to 
provide precise measurement, especially at the low end of the scale. Mathematics and 
language usage show a trend of increasing SEMs at the low end of the ability spectrum as the 
grade level increases, particularly for the on-grade item pool. SEMs for reading are large at the 
extreme ends of the scale for the on-grade item pool and at the low end for the +1-grade pool. 


The all-grade item pool is most efficient, followed by the +1-grade pool. The on-grade item pool 
is the least efficient. To achieve the same amount of information, a test using the on-grade item 
pool would need to be longer than a test using the +1-grade or all-grade item pool. However, 
relative efficiencies for reading suggest that the +1-grade pool provided comparable information 
to the all-grade item pool for on-grade simulees. In a few cases, the +1-grade item pool was 
more efficient than the all-grade pool. Results for language usage also show that the +1-grade 
item pool was almost as efficient as the all-grade pool for both on-grade and off-grade simulees. 


Testing students with off-grade ability using on-grade item pools results in diminished 
measurement precision. SEMs are uniformly larger and relative efficiencies are higher, 
especially for on-grade item pool, indicating that test length would need to be increased to a 
greater extent to yield the same information as using an all-grade item pool. 


On the positive side, the current on-grade item pools have an adequate number of items to 
support tests restricted to on-grade items. Almost all on-grade item pools contain more than 10 
times as many items as the test length, except for language usage Grade 2 and Grade 8 item 
pools. Further enhancement needs to focus on item distribution for providing more effective 
measurement. 


Tests produced from item pools with grade-level restrictions had less information and fewer 
items per instructional area. However, these limitations can be overcome by the following during 
development of on-grade items: 


e Adding more items to the low and high ends of each grade’s ability spectrum, especially 
to those flagged in this study 

e Adding items particularly to those weak instructional areas (if content balancing is part of 
the test design) 


Once more grade-level items are developed and field tested, it is recommended to have new 


tests go through a standardized test validation procedure. Given test specifications (e.g., 
number of administrations, longitudinal constraint), we can provide additional suggestions. 
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Appendix A: SEM and Reliability Efficiency 


Appendix A: SEM and Reliability Efficiency 


Table A.1. SEM and Relative Efficiency—Mathematics 


| On-Grade Simulee Off-Grade Simulee 
SEM Relative Efficiency SEM Relative Efficiency 
True On- +1- All- True On- +1- All- 

Grade RIT Grade Grade Grade RE1 RE2 RIT | Grade Grade Grade RE1 RE2 
151 2.98 2.96 2.95 1.02 1.00 133 4.24 4.06 3.39 1.49 1.37 
159 2.96 2.96 2.96 1.00 1.00 138 3.62 3.45 3.10 1.33 1.22 
167 2.96 2.96 2.96 1.00 1.00 142 3.27 3.19 2.99 1.19 1.13 
2 183 2.97 2.97 2.97 1.00 1.00 216 3.22 2.99 2.97 1.17 1.02 
199 2.98 2.96 2.97 1.01 1.00 220 3.40 3.02 2.96 1.31 1.04 
207 3.04 2.96 2.97 1.05 1.00 224 3.69 3.07 2.96 1:51 1.07 

215 3.18 2.98 2.96 1.15 1.01 - - - - - - 
164 3.11 2.97 2.96 1.10 1.00 151 3.84 2.97 2.95 1.65 1.02 
172 3.00 2.96 2.97 1.02 1.00 159 3.27 2.96 2.96 1.22 1.00 
179 3.00 2.97 2.97 1.02 1.00 162 3.17 2.97 2.97 1.14 1.00 
3 194 3.06 2.96 2.96 1.07 1.00 228 3.35 2.97 2.96 1.28 1.00 
209 3.17 2.96 2.97 1.14 1.00 232 3.41 2.98 2.97 1.32 1.01 
216 3.22 2.97 2.96 1.19 1.01 236 3.51 3.00 2.96 1.39 1.02 

224 3.29 2.97 2.96 1.23 1.00 - - - - - - 
174 3.13 2.96 2.96 1.12 1.00 164 3.55 3.02 2.96 1.42 1.04 
181 3.02 2.96 2.97 1.03 1.00 168 3.32 2.98 2.97 1.25 1.01 
189 2.97 2.97 2.97 1.00 1.00 172 3.18 2.96 2.97 1.15 1.00 
4 205 2.96 2.96 2.97 1.00 1.00 239 3.07 3.00 2.97 1.07 1.02 
220 2.96 2.97 2.96 1.00 1.00 243 3.21 3.09 2.96 1.17 1.09 
228 2.97 2.97 2.96 1.00 1.00 247 3.42 3.21 2.98 1.30 1.15 

236 3.00 2.98 2.96 1.03 1.01 - - - - - - 
180 3.57 3.01 2.97 1.43 1.03 174 4.05 3.09 2.96 1.82 1.09 
188 3.23 2.97 2.97 1.18 1.00 176 3.89 3.05 2.97 1.68 1.06 
197 3.04 2.96 2.96 1.05 1.00 179 3.65 3.02 2.97 1.50 1.04 
5 214 3.01 2.96 2.97 1.03 1.00 248 3.65 3.11 2.99 1.49 1.09 
230 3.13 2.97 2.96 1.11 1.00 249 3.69 3.13 2.99 1.51 1.09 
239 3.35 2.98 2.96 1.27 1.01 251 3.76 3.18 3.01 1.54 1.11 

247 3.62 3.09 2.98 1.46 1.07 - - - - - - 
183 3.21 3.10 2.96 1.17 1.09 180 3.34 3.18 2.96 1.26 1.15 
192 3.02 2.99 2.97 1.03 1.02 181 3.30 3.15 2.96 1.23 1.13 
200 2.97 2.97 2.96 1.01 1.01 182 3.26 3.12 2.96 1.20 1.10 
6 218 2.97 2.96 2.96 1.01 1.00 254 3.08 2.96 2.96 1.08 1.00 
234 2.96 2.96 2.96 1.00 1.00 257 3.15 2.96 2.96 113 1.00 
243 2.96 2.97 2.97 1.00 1.00 260 3.27 2.97 2.96 1.21 1.00 

251 3.01 2.97 2.97 1.03 1.00 - - - - - - 
187 3.76 3.06 2.96 1.60 1.07 183 4.09 3.16 2.96 1.86 1.13 
196 3.42 2.99 2.96 1:33 1.02 184 3.99 3.13 2.97 1.77 1.11 
205 3.11 2.97 2.97 1.10 1.00 185 3.89 3.11 2.97 1.69 1.09 
7 223 2.96 2.96 2.96 1.00 1.00 262 2.98 2.96 2.96 1.01 1.00 
242 2.96 2.97 2.97 1.00 1.00 264 3.01 2.96 2.96 1.03 1.00 
251 2.97 2.96 2.97 1.00 1.00 267 3.07 2.98 2.96 1.07 1.01 

260 2.97 2.96 2.96 1.00 1.00 - - - - - - 
190 5.20 3.30 2.96 2.83 1.23 187 5.80 3.44 2.96 3.42 1.33 
200 3.93 3.05 2.96 1.73 1.06 189 5.40 3.33 2.96 3.00 1.26 
209 3.42 2.98 2.97 1.32 1.01 190 5.15 3.30 2.97 2.78 1.23 
8 229 2.97 2.96 2.96 1.00 1.00 268 3.17 2.96 2.96 1.14 1.00 
248 2.96 2.96 2.96 1.00 1.00 269 3.20 2.96 2.96 1.16 1.00 
258 2.99 2.96 2.97 1.02 1.00 270 3.23 2.96 2.96 1.18 1.00 

267 3.14 2.96 2.96 1.12 1.00 - - - - - - 
Min. 2.96 2.96 2.95 1.00 1.00 - 2.98 2.96 2.95 1.01 1.00 
Max. 5.20 3.30 2.98 2.83 1.23 - 5.80 4.06 3.39 3.42 1.37 
Mean 3.16 2.98 2.96 1.14 1.01 - 3.59 3.10 2.98 1.46 1.08 
SD 0.37 0.06 0.00 0.30 0.04 - 0.62 0.20 0.07 0.51 0.09 


Note. Test information is 100/SEM?. 
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Appendix A: SEM and Reliability Efficiency 


Table A.2. SEM and Relative Efficiency—Reading 


On-Grade Simulee Off-Grade Simulee 
SEM Relative Efficiency SEM Relative Efficiency 
True On- +1- All- True On- +1- All- 

Grade RIT Grade Grade Grade RE1 RE2 RIT Grade Grade Grade RE1 RE2 
145 4.44 4.14 3.87 1.3 1.13 134 6.67 6.00 5.52 1.39 1.16 
154 3.79 3.57 3.47 1.19 1.05 138 5.67 5.15 4.77 1.34 1.15 
163 3.56 3.37 3.36 1.12 1.01 142 4.93 4.49 4.17 1.34 1.14 
2 180 3.39 3.35 3.34 1.03 1.00 218 4.64 3.47 3.34 1.84 1.08 
198 3.41 3.34 3.34 1.04 1.00 223 5.69 3.82 3.35 2.54 1.27 
207 3.65 3.34 3.34 1.18 1.00 227 6.52 4.26 3.34 3.27 1.56 

216 4.35 3.41 3.34 1.63 1.04 = - - - = = 
157 3.76 3.48 3.42 1.18 1.03 145 5.72 4.24 3.87 1.91 1.18 
165 3.38 3.36 3.35 1.01 1.01 150 4.63 3.79 3.59 1.54 1.11 
174 3.33 3.34 3.34 1.00 1.00 154 3.99 3.58 3.47 1.28 1.06 
3 192 3.34 3.35 3.35 0.99 1.00 229 4.64 3.73 3.34 1.77 1.23 
209 3.34 3.34 3.35 0.99 0.99 232 5.15 4.04 3.34 2.13 1.40 
218 3.47 3.34 3.34 1.07 1.00 234 5.65 4.24 3.34 2.47 1.54 

227 4.27 3.58 3.34 1.56 1.14 = - - - = = 
167 3.4 3.32 3.34 1.03 0.99 157 3.97 3.62 3.42 1.30 1.10 
175 3.33 3.34 3.35 0.99 1.00 161 3.62 3.42 3.37 1.14 1.03 
184 3.35 3.35 3.35 1.00 1.00 165 3.44 3.34 3.35 1.05 1.00 
4 201 3.34 3.34 3.34 1.00 1.00 236 4.67 4.03 3.36 1.77 1.38 
218 3.35 3.33 3.34 1.00 0.99 238 5.13 4.33 3.38 2.05 1.54 
226 3.55 3.37 3.34 1.12 1.01 241 5.88 4.83 3.47 2.48 1.78 

234 4.35 3.83 3.34 1.61 1.27 - - - - = = 
174 3.47 3.33 3.34 1.08 1.00 167 3.88 3.39 3.34 1.31 1.03 
183 3.36 3.34 3.35 1.01 0.99 170 3.63 3.35 3.34 1.17 1.00 
181 3.35 3.34 3.35 1.00 1.00 172 3.55 3.34 3.34 1.12 1.00 
5 208 3.33 3.34 3.35 0.99 1.00 242 5.41 3.64 3.5 2.08 1.06 
224 3.35 3.34 3.34 1.01 1.00 244 5.93 3.81 3.61 2.29 1.10 
232 3.73 3.34 3.34 1.22 1.00 245 6.18 3.91 3.69 2.40 1.10 

241 5.06 3.57 3.46 1.92 1.05 - - - - = - 
180 3.80 3.37 3.34 1.26 1.02 174 4.51 3.47 3.33 1.71 1.08 
188 3.40 3.34 3.35 1.03 1.00 176 4.22 3.42 3.33 1.52 1.05 
196 3.33 3.34 3.34 0.99 1.00 179 3.85 3.38 3.34 1.30 1.02 
6 212 3.34 3.34 3.35 0.99 0.99 246 4.04 3.59 3.34 1.40 1.14 
229 3.34 3.34 3.35 0.99 1.00 248 4.25 3.76 3.34 1.55 1.24 
237 3.41 3.34 3.34 1.03 1.00 249 4.43 3.84 3.35 1.63 1.29 

245 3.91 3.53 3.34 1.32 1.10 - - - - - - 
184 3.89 3.51 3.34 1.31 1.09 180 4.37 3.78 3.34 1.60 1.25 
192 3.40 3.34 3.35 1.03 0.99 182 4.08 3.62 3.34 1.42 1.16 
200 3.33 3.34 3.35 0.99 0.99 183 3.99 3.55 3.34 1.37 1.12 
7 216 3.34 3.34 3.34 1.00 1.00 250 4.15 3.62 3.35 1.47 1.15 
233 3.34 3.34 3.34 1.00 1.00 252 4.38 3.75 3.37 1.59 1.21 
241 3.43 3.34 3.34 1.05 1.00 253 4.57 3.89 3.4 1.68 1.28 

249 3.99 3.54 3.35 1.38 1.10 - - - - = = 
188 3.84 3.53 3.35 1.28 1.11 184 4.30 3.83 3.34 1.56 1.28 
196 3.45 3.33 3.34 1.06 0.99 186 4.01 3.68 3.34 1.40 1.19 
204 3.35 3.34 3.35 1.00 0.99 187 3.95 3.62 3.34 1.35 1.15 
8 220 3.34 3.34 3.34 1.00 1.00 254 4.09 3.84 3.42 1.37 1.23 
237 3.33 3.33 3.35 0.99 0.99 255 4.24 3.95 3.44 1.45 1.28 
245 3.43 3.36 3.34 1.05 1.01 256 4.41 4.04 3.48 1.52 1.30 

253 3.98 3.74 3.4 1.32 1.19 - - = = - - 
Min. S88) 3.32 3.34 0.99 0.99 - 3.44 3.34 3.33 1.05 1.00 
Max. 5.06 4.14 3.87 1.92 ab27, = 6.67 6.00 5.52 3.27 1.78 
Mean 3.59 3.41 3.36 1.13 1.03 - 4.64 3.87 3.50 1.66 1.20 
SD 0.38 0.15 0.08 0.20 0.06 - 0.84 0.52 0.41 0.47 0.17 


Note. Test information is 100/SEM2. 
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Appendix A: SEM and Reliability Efficiency 


Table A.3. SEM and Relative Efficiency—Language Usage 


On-Grade Simulee Off-Grade Simulee 
SEM Relative Efficiency SEM Relative Efficiency 
True On- +1- All- True On- +1- All- 

Grade RIT Grade Grade Grade RE1 RE2 RIT Grade Grade Grade RE1 RE2 
147 4.55 3.77 3.65 1.47 1.06 134 7.97 6.54 6.14 1.69 1.12 
155 3.53 3.17 3.12 1.26 1.03 138 6.89 5.38 5.17 1.66 1.06 
164 3.1 2.98 2.97 1.09 1.01 142 5.70 4.54 4.32 1.58 1.08 
2 180 2.96 2.96 2.96 1.00 1.00 216 3.67 3.00 2.97 1.49 1.02 
197 2.97 2.96 2.96 1.01 1.00 220 4.12 3.08 2.97 1.82 1.08 
206 3.09 2.96 2.96 1.08 1.00 224 4.83 3.22 2.97 2.39 1.17 

214 3.45 2.98 2.96 1.33 1.01 - - - - - = 
160 3.12 3.03 3.01 1.07 1.01 155 3.47 3.24 3.14 1.20 1.06 
168 2.95 2.95 2.96 0.99 1.00 157 3.30 3.13 3.07 1.14 1.04 
176 2.96 2.96 2.96 1.00 1.00 159 3.17 3.05 3.03 1.08 1.01 
3 192 2.96 2.96 2.96 1.00 1.00 226 3.30 3.03 2.96 1.23 1.04 
208 2.96 2.96 2.97 0.99 1.00 228 3.41 3.06 2.96 1.31 1.07 
216 2.99 2.96 2.96 1.02 1.00 231 3.60 3.15 2.96 1.45 1.13 

224 3.21 3.00 2.97 1.17 1.02 = - - - - - 
170 2.99 2.95 2.96 1.02 1.00 160 3.48 3.06 3.01 1.30 1.03 
177 2.96 2.96 2.96 1.00 1.00 165 3.13 2.96 2.97 1.11 1.00 
185 2.97 2.97 2.97 1.00 1.00 169 3.01 2.95 2.95 1.04 1.00 
4 201 2.96 2.96 2.96 1.00 1.00 232 3.37 3.01 2.96 1.28 1.03 
216 2.96 2.96 2.96 1.00 1.00 234 3.52 3.06 2.96 1.39 1.06 
224 3.02 2.97 2.97 1.03 1.00 236 3.75 3.11 2.96 1.55 1.10 

231 3.29 3.00 2.96 1.22 1.03 - - - - - - 
177 3.13 2.96 2.96 1.11 1.00 170 3.68 2.99 2.96 1.50 1.02 
184 2.95 2.96 2.96 0.99 1.00 173 3.41 2.96 2.96 1.30 1.00 
192 2.96 2.96 2.96 1.00 1.00 176 3.17 2.96 2.96 1.14 1.00 
5 207 2.96 2.96 2.96 1.00 1.00 237 3.47 2.98 2.96 1.36 1.01 
222 2.98 2.96 2.97 1.01 1.00 239 3.66 3.01 2.96 1.49 1.04 
229 3.07 2.96 2.96 1.07 1.00 241 3.92 3.08 2.96 1.68 1.08 

236 3.39 2.97 2.96 1.30 1.01 = - - - = = 
181 3.47 3.00 2.96 1.35 1.02 177 3.88 3.10 2.96 1.65 1.09 
189 3.06 2.95 2.96 1.07 1.00 179 3.66 3.03 2.96 1.49 1.04 
196 2.98 2.97 2.97 1.01 1.00 180 3.55 3.01 2.96 1.41 1.03 
6 211 2.96 2.96 2.97 1.00 0.99 242 3.40 3.07 2.96 1.30 1.08 
226 2.97 2.96 2.97 1.00 1.00 243 3.46 3.11 2.96 1.34 1.09 
233 3.03 2.96 2.96 1.04 1.00 244 3.55 3.15 2.96 1.40 1.12 

241 3.32 3.03 2.96 1.25 1.05 = - - - - - 
185 3.56 3.13 2.96 1.42 1.11 181 3.89 3.30 2.96 1.68 1.23 
193 3.14 2.98 2.96 1.12 1.01 183 3.69 3.20 2.96 1.53 1.16 
200 2.99 2.97 2.96 1.02 1.00 184 3.63 3.17 2.96 1.48 1.14 
7 215 2.96 2.96 2.97 0.99 1.00 245 3.38 3.11 2.97 1.28 1.09 
230 2.96 2.97 2.96 1.00 1.00 247 3.58 3.18 3.00 1.39 1.12 
237 3.03 2.98 2.96 1.05 1.01 248 3.68 3.22 3.02 1.44 1.13 

244 3.32 3.07 2.97 1.24 1.07 = - - - = = 
189 4.02 3.22 2.96 1.76 1.18 185 4.72 3.44 2.96 2.28 1.34 
196 3.36 3.00 2.96 1.27 1.03 186 4.51 3.38 2.96 2.12 1.29 
204 3.05 2.96 2.96 1.06 1.00 188 4.13 3.27 2.97 1.84 1.20 
8 218 2.97 2.96 2.97 1.01 1.00 249 3.56 3.21 3.05 1.34 1.10 
233 3.05 2.96 2.96 1.06 1.00 250 3.67 3.28 3.07 1.39 1.13 
241 3.13 2.97 2.96 1.12 1.01 251 3.77 3.34 3.12 1.43 1.13 

248 3.48 3.16 3.02 1.31 1.09 - - - - - - 
Min. 2.95 2.95 2.96 0.99 0.99 - 3.01 2.95 2.95 1.04 1.00 
Max. 4.55 3.77 3.65 1.76 1.18 - 7.97 6.54 6.14 2.39 1.34 
Mean 3.15 3.00 2.98 1.11 1.01 - 3.87 3.29 3.14 1.48 1.09 
SD 0.30 0.13 0.10 0.16 0.03 - 0.95 0.67 0.62 0.29 0.08 


Note. Test information is 100/SEM2. 
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Appendix B: Bias and RMSE 


Table B.1. Bias and RMSE—Mathematics 


Appendix B: Bias and RMSE 


On-Grade Simulee 


Off-Grade Simulee 


Bias RMSE Bias RMSE 
True On- +1- All- On- +1- All- True On- +1- All- On- +1- All- 
Grade RIT Grade Grade Grade | Grade Grade Grade RIT Grade Grade Grade | Grade Grade Grade 
151 -0.12 -0.04 -0.14 2.35 2.35 2.22 133 0.41 0.51 0.07 3.48 3.34 2.61 
159 -0.10 0.05 -0.11 2.32 2.26 2.27 138 0.21 0.07 0.00 2.99 2.78 2.55 
167 -0.10 0.06 0.07 2.39 2.34 2.43 142 -0.20 0.12 -0.15 2.60 2.78 2.41 
2 183 0.01 0.12 0.00 2.56 2.37 2.38 246 -0.13 -0.11 -0.10 2.57 2.47 2.42 
199 -0.10 -0.01 0.01 2.40 2.30 2.25 220 -0.31 0.08 0.07 2.57 2.34 2.32 
207 -0.34 0.36 -0.12 2.44 2.47 2.54 224 -0.36 -0.14 0.13 2.92 2.53 2.40 
215 -0.03 -0.04 0.08 2.52 2.45 2.44 - = = = - = - 
164 -0.14 -0.01 -0.31 2.46 2.36 2.33 151 0.39 -0.25 -0.14 2.97 2.29 2.22 
172 0.00 0.07 0.09 2.22 2.36 2.38 159 0.09 -0.14 -0.11 2.50 2.38 2.27 
179 -0.17 -0.21 0.12 2.37 2.25 2.52 162 -0.07 -0.13 0.08 2.47 2.54 2.27 
3 194 -0.25 0.02 -0.06 2.49 2.38 2.37 228 -0.04 -0.04 -0.02 2.70 2.36 2.42 
209 0.09 -0.11 -0.11 2.66 2.50 2.43 232 0.02 -0.07 0.08 2.63 2.47 2.31 
216 0.01 -0.10 -0.15 2.50 2.20 2.20 236 -0.17 -0.02 -0.04 2.76 2.34 2.42 
224 0.23 -0.04 -0.08 2.69 2.41 2.51 - = - - - - - 
174 0.00 0.16 -0.02 2.49 2.26 2.36 164 0.02 0.10 -0.31 2.86 2.57 2.33 
181 -0.25 0.03 0.15 2.49 2.41 2.20 168 0.05 0.15 -0.14 2.77 2.45 2.52 
189 -0.03 -0.06 0.01 2.40 2.34 2.33 172 0.16 -0.04 0.05 2.54 2.51 2.36 
4 205 -0.18 0.01 -0.23 2.40 2.31 2.39 239 -0.18 0.07 0.08 2.36 2.42 2.24 
220 0.14 0.00 0.07 2.40 2.33 2.32 243 -0.34 -0.20 0.18 2.56 2.46 2.30 
228 -0.03 -0.08 -0.09 2.39 2.51 2.34 247 -0.08 -0.09 -0.04 2.58 2.57 2.29 
236 -0.25 -0.15 -0.06 2.21 2.33 2.30 - = = = - - - 
180 0.12 0.14 -0.19 2.87 2.36 2.46 174 0.41 0.20 -0.02 3.25 2.34 2.36 
188 0.13 -0.02 0.00 2.76 2.31 2.37 176 0.57 -0.14 0.13 3.18 2.34 2.38 
197 0.06 -0.02 -0.06 2.41 2.45 2.35 179 0.39 0.19 0.12 2.84 2.43 2.52 
5 214 0.29 -0.14 0.12 2.33 2.46 2.36 248 -0.41 -0.20 0.06 2.97 2.48 2.57 
230 -0.15 0.06 0.00 2.59 2.49 2.42 249 -0.35 -0.16 -0.01 3.05 2.41 2.37 
239 -0.25 0.08 0.14 2.83 2.53 2.35 251 0.04 -0.04 -0.08 3.10 2.57 2.46 
247 -0.40 0.00 -0.15 3.03 2.55 2.46 - - = = - - - 
183 -0.17 -0.05 0.05 2.54 2.46 2.59 180 -0.08 0.02 -0.22 2.58 2.58 2.41 
192 -0.09 0.06 -0.06 2.47 2.51 2.24 181 0.12 0.04 0.00 2.66 2.46 2.40 
200 -0.20 0.15 -0.29 2.35 2.28 2.37 182 -0.07 -0.20 -0.03 2.64 2.44 2.42 
6 218 -0.02 -0.24 -0.12 2.40 2.48 2.29 254 -0.31 -0.30 0.10 2.33 2.35 2.53 
234 0.19 -0.03 -0.05 2.31 2.45 2.43 257 -0.01 -0.01 -0.12 2.51 2.31 2.41 
243 0.02 0.04 -0.16 2.29 2.40 2.27 260 0.06 -0.09 -0.01 2.65 2.40 2.41 
251 -0.10 -0.02 0.22 2.17 2.35 2.28 - - = = - - - 
187 0.35 -0.03 -0.02 2.87 2.42 2.33 183 0.56 0.06 0.05 3.39 2.47 2.59 
196 0.13 0.00 -0.26 2.76 2.33 2.43 184 0.34 0.15 -0.13 3.42 2.49 2.38 
205 0.06 -0.16 0.24 2.49 2.38 2.40 185 0.16 0.13 -0.04 2.85 2.56 2.55 
7 223 -0.13 -0.25 -0.24 2.32 2.36 2.46 262 -0.05 -0.20 -0.06 2.38 2.47 2.46 
242 0.13 0.16 0.23 2.37 2.39 2.33 264 0.10 0.09 0.06 2.45 2.43 2.29 
251 0.12 0.01 0.01 2.38 2.43 2.36 267 -0.08 0.13 -0.01 2.53 2.42 2.39 
260 -0.03 -0.15 -0.05 2.40 2.37 2.40 - = = = - - - 
190 1.00 -0.22 -0.10 4.37 2.83 2.46 187 1.23 0.09 -0.02 4.69 2.91 2.33 
200 0.29 -0.11 0.11 3.18 2.49 2.23 189 1.03 -0.19 -0.02 4.47 2.75 2.43 
209 0.16 -0.33 -0.03 2.79 2.40 2.33 190 0.67 -0.04 -0.02 4.38 2.70 2.48 
8 229 -0.15 -0.07 0.16 2.35 2.38 2.33 268 -0.10 0.03 -0.10 2.56 2.52 2.38 
248 0.00 0.04 -0.01 2.55 2.28 2.39 269 -0.12 -0.04 -0.02 2.52 2.27 2.37 
258 0.03 -0.15 -0.18 2.35 2.34 2.30 270 -0.10 0.25 -0.02 2.56 2.46 2.35 
267 -0.02 -0.21 -0.17 2.38 2.29 2.33 - = = = - - - 
Min. -0.40 -0.33 -0.31 2.17 2.20 2.20 = -0.41 -0.30 -0.31 2.33 2.27 2.22 
Max. 1.00 0.36 0.24 4.37 2.83 2.59 - 1.23 0.51 0.18 4.69 3.34 2.61 
Mean -0.01 -0.03 -0.04 2.53 2.39 2.36 - 0.08 -0.01 -0.02 2.88 2.50 2.40 
SD 0.22 0.13 0.13 0.34 0.10 0.09 - 0.36 0.16 0.10 0.54 0.19 0.10 
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Table B.2. Bias and RMSE—Reading 


Appendix B: Bias and RMSE 


On-Grade Simulee 


Off-Grade Simulee 


Bias RMSE Bias RMSE 
True On- +1- All- On- +1- All- True On- +1- All- On- +1- All- 
Grade RIT Grade Grade Grade | Grade Grade Grade RIT Grade Grade Grade | Grade Grade Grade 
145 0.19 0.44 -0.04 3.57 3.36 3.19 134 1.89 1.26 1.19 5.77 4.82 4.58 
154 -0.24 0.18 -0.29 3.20 3.00 2.72 138 1.13 1.03 0.92 4.91 4.17 3.93 
163 -0.04 0.21 0.26 2.93 2.70 2.93 142 0.95 0.62 0.39 4.11 3.69 3.27 
2 180 0.18 0.20 -0.33 2.85 2.72 2.53 218 -0.51 0.03 -0.21 3.63 2.73 2.72 
198 0.14 0.10 -0.29 2.77 2.70 2.64 223 -1.44 -0.38 0.11 4.87 3.05 2.50 
207 -0.18 -0.03 -0.10 3.04 2.47 2.67 227 -1.31 -0.41 0.13 5.24 3.08 2.75 
216 -0.18 -0.05 -0.13 3.34 2.56 2.63 - = = = - - - 
157 0.62 -0.04 0.12 2.97 2.83 2.97 145 1.26 0.33 -0.04 4.67 3.48 3.19 
165 0.18 0.20 -0.21 2.68 2.79 2.74 150 0.69 0.19 0.05 4.05 3.01 2.80 
174 -0.21 0.12 -0.21 2.53 2.83 2.73 154 0.02 0.12 0.09 3.23 2.99 2.63 
3 192 0.20 -0.02 0.22 2.72 2.72 2.56 229 -0.69 -0.48 0.15 3.93 2.89 2.49 
209 0.29 0.23 -0.14 2.55 2.76 2.69 232 -0.68 -0.53 0.25 4.15 3.29 2.64 
218 0.13 -0.34 0.05 2.84 2.80 2.81 234 -0.92 -0.30 -0.12 4.57 3.28 2.62 
227 -0.44 -0.31 0.04 3.32 2.84 2.68 - = = = - = - 
167 0.17 -0.17 0.06 2.89 2.62 2.63 157 0.49 0.13 0.12 3.41 3.06 2.97 
175 -0.08 -0.01 -0.01 2.64 2.59 2.78 161 0.20 0.01 -0.37 2.91 2.57 2.67 
184 0.19 0.03 -0.27 2.64 2.67 2.48 165 0.06 -0.12 0.02 2.65 2.67 2.73 
4 201 -0.38 0.05 -0.20 2.81 2.79 2.60 236 -0.65 -0.38 0.10 3.66 3.25 2.72 
218 -0.19 0.28 -0.32 2.76 2.70 2.74 238 -1.25 -0.65 0.04 4.27 3.56 2.67 
226 -0.16 0.12 -0.02 2.94 2.74 2.61 241 -1.86 -0.79 -0.26 4.78 3.82 2.86 
234 -0.75 -0.40 0.05 3.44 3.19 2.74 - = = = - = - 
174 0.05 -0.11 -0.10 2.89 2.73 2.53 167 0.38 0.06 0.06 3.25 2.73 2.63 
183 0.12 0.28 -0.11 2.74 2.52 2.70 170 0.00 -0.01 -0.23 2.97 2.74 2.65 
181 -0.23 0.04 0.08 2.57 2.74 2.63 172 0.24 0.02 -0.07 2.81 2.67 2.55 
5 208 -0.10 0.10 -0.17 2.71 2.87 2.67 242 -1.51 -0.09 -0.23 4.51 3.01 2.61 
224 -0.11 0.01 0.19 2.72 2.63 2.72 244 -1.73 -0.28 -0.13 4.88 3.18 2.96 
232 -0.31 0.02 0.15 3.05 2.72 2.63 245 -1.94 -0.40 -0.39 5.17 3.21 3.16 
241 -0.92 -0.01 0.21 4.12 2.93 2.75 - = - = - - - 
180 0.52 0.23 0.02 3.23 2.70 2.81 174 0.68 0.35 -0.03 3.83 2.79 2.62 
188 0.17 0.04 0.11 2.64 2.81 2.74 176 0.57 -0.13 -0.14 3.45 2.90 2.69 
196 -0.05 -0.18 0.01 2.59 2.58 2.73 179 0.22 -0.09 0.06 3.03 2.77 2.65 
6 212 -0.22 0.07 0.08 2.76 2.83 2.66 246 -0.45 0.01 0.12 3.21 2.88 2.57 
229 -0.01 0.03 -0.11 2.71 2.63 2.71 248 -0.34 -0.37 0.23 3.31 3.07 2.55 
237 0.19 0.10 -0.03 2.83 2.76 2.73 249 -0.41 -0.37 0.01 3.79 3.01 2.78 
245 -0.11 0.04 0.07 3.36 2.86 2.72 - - = = - = - 
184 0.30 -0.02 0.02 3.15 2.84 2.71 180 0.42 0.27 0.02 3.75 3.05 2.81 
192 -0.05 0.07 0.03 2.75 2.61 2.71 182 0.19 0.04 0.04 3.36 2.82 2.64 
200 -0.16 0.07 0.05 2.71 2.75 2.69 183 0.43 -0.09 -0.08 3.23 2.90 2.80 
7 216 -0.03 0.18 0.26 2.61 2.70 2.79 250 -0.72 -0.51 0.01 3.33 2.97 2.66 
233 -0.01 0.29 0.08 2.48 2.78 2.88 252 -0.46 -0.27 0.18 3.39 2.89 2.56 
241 -0.10 -0.14 -0.12 2.80 2.82 2.47 253 -0.61 -0.81 -0.10 3.76 3.25 2.65 
249 -0.40 -0.06 0.09 3.00 2.80 2.63 - = = = - = - 
188 0.06 -0.12 -0.09 3.16 2.83 2.93 184 0.26 0.01 0.02 3.62 3.02 2.71 
196 0.11 -0.04 0.37 2.63 2.63 2.83 186 0.05 0.20 -0.02 2.99 3.02 2.67 
204 -0.17 -0.07 0.05 2.79 2.74 2.50 187 0.34 0.25 -0.03 3.25 3.11 2.73 
8 220 0.01 0.15 -0.04 2.60 2.84 2.57 254 -0.44 -0.27 -0.14 3.30 3.07 2.73 
237 0.11 0.07 0.12 2.63 2.74 2.74 255 -0.76 -0.48 0.05 3.48 3.24 2.64 
245 -0.48 0.12 -0.17 2.75 2.72 2.75 256 -1.00 -0.31 -0.23 3.68 3.13 2.64 
253 -0.44 -0.17 -0.09 3.19 2.97 2.78 - = = = - - - 
Min. -0.92 -0.40 -0.33 2.48 2.47 2.47 - -1.94 -0.81 -0.39 2.65 225 2.49 
Max. 0.62 0.44 0.37 4.12 3.36 3.19 = 1.89 1.26 1.19 5.77 4.82 4.58 
Mean -0.06 0.04 -0.02 2.89 2.76 2.71 - -0.22 -0.08 0.04 3.81 3.12 2.80 
SD 0.28 0.17 0.16 0.32 0.15 0.13 - 0.87 0.42 0.28 0.75 0.42 0.38 
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Table B.3. Bias and RMSE—Language Usage 


Appendix B: Bias and RMSE 


On-Grade Simulee 


Off-Grade Simulee 


Bias RMSE Bias RMSE 
True On- +1- All- On- +1- All- True On- +1- All- On- +1- All- 
Grade RIT Grade Grade Grade | Grade Grade Grade RIT Grade Grade Grade | Grade Grade Grade 
147 0.74 0.35 0.19 3.68 3.06 2.87 134 2.66 1.96 1.43 6.56 5.44 5.12 
155 0.38 0.03 -0.19 2.77 2.44 2.48 138 2.40 1.07 1.15 5.95 4.61 4.29 
164 -0.09 -0.01 -0.10 2.35 2.36 2.43 142 1.35 0.76 0.53 4.64 3.77 3.42 
2 180 -0.22 -0.24 0.06 2.27 2.38 2.35 216 -0.57 -0.22 0.11 3.04 2.38 2.35 
197 -0.02 0.05 0.10 2.38 2.34 2.37 220 -0.45 -0.09 -0.08 3.44 2.49 2.28 
206 0.03 0.04 0.26 2.32 2.44 2.19 224 -0.96 -0.18 -0.25 3.92 2.55 2.49 
214 -0.03 -0.30 -0.12 2.65 2.43 2.41 - = = = - - - 
160 0.07 0.13 0.14 2.51 2.61 2.48 155 0.22 0.17 0.17 2.75 2.68 2.38 
168 -0.21 -0.06 0.27 2.38 2.43 2.41 157 0.11 0.10 -0.19 2.70 2.53 2.53 
176 0.07 -0.07 0.14 2.35 2.38 2.35 159 -0.15 -0.16 0.20 2.77 2.47 2.51 
3 192 0.05 -0.19 0.10 2.46 2.28 2.46 226 -0.04 -0.02 -0.10 2.44 2.46 2.38 
208 0.03 0.12 -0.06 2.33 2.40 2.44 228 -0.15 -0.03 0.16 2.73 2.35 2.34 
216 0.16 0.09 -0.12 2.36 2.51 2.40 231 -0.07 -0.23 0.06 3.01 2.61 2.28 
224 0.05 0.09 -0.16 2.63 2.40 2.37 - = = = - - - 
170 -0.06 -0.07 0.02 2.30 2.44 2.27 160 0.35 -0.04 0.14 2.89 2.42 2.48 
177 -0.16 0.09 -0.03 2.35 2.61 2.36 165 -0.06 0.01 0.01 2.60 2.32 2.35 
185 -0.02 0.10 0.20 2.34 2.35 2.46 169 0.03 -0.03 -0.16 2.38 2.26 2.33 
4 201 0.10 0.11 0.20 2.40 2.36 2.34 232 -0.33 0.37 0.10 2.69 2.39 2.42 
216 0.12 0.03 0.06 2.32 2.41 2.50 234 -0.20 -0.21 0.22 2.90 2.48 2.31 
224 -0.29 -0.01 -0.25 2.32 2.39 2.49 236 -0.64 -0.19 0.19 2.94 2.45 2.28 
231 -0.16 -0.12 -0.06 2.57 2.37 2.50 - = = = - - - 
177 0.17 -0.19 0.26 2.63 2.35 2.38 170 0.34 0.04 0.02 3.12 2.52 2.27 
184 0.12 -0.02 0.00 2.31 2.35 2.36 173 0.44 -0.08 0.08 2.92 2.42 2.33 
192 -0.17 0.09 -0.12 2.46 2.41 2.36 176 -0.03 -0.02 0.14 2.66 2.34 2.35 
5 207 0.13 -0.03 0.03 2.36 2.33 2.37 237 -0.11 -0.03 0.12 2.55 2.47 2.27 
222 0.03 -0.03 -0.20 2.32 2.56 2.50 239 -0.25 0.10 -0.03 2.80 2.53 2.35 
229 -0.26 -0.01 0.14 2.48 2.30 2.38 241 -0.69 -0.16 0.06 3.14 2.37 2.36 
236 -0.02 -0.11 0.05 2.67 2.47 2.32 - = = = - - - 
181 0.09 0.26 -0.16 2.75 2.49 2.45 177 0.35 0.27 0.26 3.17 2.68 2.38 
189 -0.25 0.03 0.18 2.54 2.30 2.20 179 0.26 0.07 -0.05 3.04 2.34 2.40 
196 -0.02 -0.18 -0.03 2.41 2.34 2.30 180 0.05 -0.12 0.01 2.87 2.47 2.36 
6 211 0.11 -0.17 -0.06 2.65 2.32 2.30 242 -0.53 -0.08 0.03 2.72 2.62 2.27 
226 -0.01 0.20 -0.05 2.40 2.44 2.38 243 -0.40 -0.19 -0.02 2.70 2.37 2.36 
233 -0.30 -0.16 -0.19 2.39 2.29 2.45 244 -0.45 -0.08 0.15 2.81 2.53 2.24 
241 -0.29 0.04 -0.03 2.69 2.46 2.38 - - = = - - - 
185 0.33 0.08 -0.15 3.05 2.42 2.37 181 0.55 0.18 -0.16 3.22 2.62 2.45 
193 0.20 -0.15 -0.03 2.5-7 2.39 2.47 183 0.22 0.14 0.03 2.87 2.55 2.22 
200 0.29 0.09 -0.12 2.35 2.42 2.37 184 0.37 0.30 -0.09 3.02 2.49 2.28 
7 215 -0.14 -0.03 -0.04 2.36 2.32 2.31 245 -0.04 -0.18 0.17 2.61 2.49 2.39 
230 0.11 0.08 -0.07 2.46 2.38 2.47 247 -0.29 0.05 -0.04 2.96 2.51 2.29 
237 -0.03 0.17 -0.05 2.40 2.33 2.29 248 -0.37 0.20 0.04 2.92 2.56 2.48 
244 -0.10 -0.01 0.15 2.75 2.38 2.43 - = = = - = - 
189 0.35 0.15 -0.10 3.28 2.64 2.42 185 0.83 0.33 -0.15 3.94 2.80 2.37 
196 0.00 -0.08 0.19 2.93 2.35 2.27 186 0.64 0.26 0.00 3.81 2.92 2.43 
204 -0.08 -0.40 0.19 2.40 2.39 2.37 188 0.17 0.21 -0.10 3.30 2.54 2.35 
8 218 -0.07 -0.03 -0.04 2.35 2.48 2.33 249 -0.21 -0.05 -0.13 2.76 2.57 2.62 
233 -0.13 0.15 -0.14 2.48 2.26 2.40 250 -0.46 -0.27 0.02 3.05 2.60 2.41 
241 0.07 0.06 0.06 2.50 2.29 2.36 251 -0.47 -0.03 0.00 3.07 2.78 2.49 
248 -0.18 -0.15 0.02 2.82 2.73 2.41 - = - = - - - 
Min. -0.30 -0.40 -0.25 22, 2.26 2.19 - -0.96 -0.27 -0.25 2.38 2.26 2.22 
Max. 0.74 0.35 0.27 3.68 3.06 2.87 = 2.66 1.96 1.43 6.56 5.44 S12 
Mean 0.01 0.00 0.01 2.53 2.42 2.39 - 0.08 0.09 0.10 3.15 2.66 2.51 
SD 0.20 0.14 0.14 0.27 0.13 0.10 - 0.71 0.39 0.31 0.83 0.59 0.54 
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Table C.1. Item Exposure Rates for On-Grade Simulees—Mathematics 


Appendix C: Item Exposure Rates 


Appendix C: Item Exposure Rates 


+1 Grade Item Pool 


All-Grade Item Pool 


True 
SG RIT G1 G2 G3 G4 G5 G6 G7 G8 G9 K G1 G2 G3 G4 G5 G6 G7 G8 G9 
151 19.6 72.4 8.0 - - - - - -— | 37.1 21.1 37.2 4.1 0.5 - - - - - 
159 | 17.9 686 13.5 - - - - - -—| 275 274 36.9 6.5 1.6 0.2 - - - = 
167 | 17.7 60.5 21.9 - - - - - -/| 198 29.0 361 11.0 3.6 0.5 0.1 - - = 
2 183 | 13.9 38.7 47.4 - - - - - -/ 126 14.3 27.2 29.3 12.2 3.6 1.0 - - - 
199 9.2 29.5 61.4 - - - - - - 3.3 6.7 143 29.9 324 10.9 2.2 0.2 - = 
207 6.7 27.0 66.3 - - - - - - 0.7 4.4 8.8 226 403 15.2 6.5 1.3 0.1 0.1 
215 3.1. 27.4 69.5 = - - - - - 0.1 1.6 49 144 391 196 15.9 3.7 0.3 0.3 
164 - 78.5 16.4 5.1 - - - - -— | 224 29.0 36.3 9.1 2.7 0.5 - = - - 
172 - 65.1 25.8 9.1 - - - - -| 164 260 353 15.7 5.3 1.0 0.3 - - - 
179 - 490 364 14.6 - - - - -| 142 184 31.0 249 8.7 2:3 0.5 = - = 
3 194 - 250 39.9 365.1 - - - - -| 6.0 7.7 18.5 31.7 25.9 8.5 1.7 0.1 - - 
209 - 10.7 29.4 60.0 - - - - - 0.4 3.8 7.8 20.7 406 16.3 8.3 1.8 0.1 0.2 
216 - 68 244 68.8 - - - - - 0.1 1.3 43 135 384 203 17.4 4.0 0.5 0.4 
224 - 41 23.8 72.1 - - - - - - 0.2 2.1 94 28.8 23.7 26.0 7.1 1.9 0.9 
174 - - 69.5 27.3 3.2 - - - -| 151 23.9 344 18.7 6.1 1.3 0.4 - - = 
181 - - 658 28.9 5.3 - - - -/| 13.7 16.3 295 269 10.3 2.7 0.7 - - = 
189 - - 56.9 343 8.8 - - - - 9.3 96 22.1 324 19.3 6.0 1.3 - - - 
4 205 - - 33.2 49.2 17.6 - - - - 1.0 5.0 10.2 246 38.9 14.4 5.0 0.9 - 0.1 
220 - - 164 53.8 29.8 - - - - - 0.5 3.1. 11.1 33.6 22.1 22.7 5.4 0.9 0.6 
228 - - 125 48.7 38.9 - - - - - 0.1 1.4 7.8 244 249 28.0 8.7 3.3 1.4 
236 - - 73 47.1 456 - - - - - - 0.4 46 185 253 29.5 12.9 6.0 3.0 
180 - - - 81.6 13.0 5.5 - - - | 13.9 17.0 303 25.9 9.7 2.7 0.6 - - - 
188 - - - 77.0 15.6 7.4 - - -/ 100 10.3 232 31.9 17.9 5.6 1.2 - - - 
197 - - - 748 16.8 8.5 - - -| 44 6.9 15.9 30.7 30.5 9.6 1.9 0.1 - - 
5 214 - - - 63.3 246 12.1 - - - 0.1 1.9 5.5 15.8 394 19.1 14.3 3.4 0.3 0.3 
230 - - - 45.7 349 19.5 - - - - - 1.2 6.9 229 25.2 28.2 9.7 4.1 1.7 
239 - - - 37.2 39.0 23.8 - - - - - 0.2 3.6 164 25.7 29.8 14.1 6.5 3.6 
247 - - - 33.3 40.5 26.3 - - - - - - 1.9 104 255 31.1 19.7 6.6 4.8 
183 - - - - 204 64.5 15.1 - - - - - 61.9 17.0 6.8 10.8 2.4 0.5 0.6 
192 - - - - 184 644 17.2 - - - - - 48.2 22.1 7.3 13.4 4.8 1.6 2.7 
200 - - - - 166 62.2 21.2 - - - - - 33.0 27.2 9.5 15.8 8.5 2.7 3.3 
6 218 - - - - 143 47.1 38.6 - - - - - 8.9 20.1 12.1 21.7 19.2 9.2 8.7 
234 - - - - 107 45.2 44.1 - - - - - 2.2 5.9 10.9 23.3 22.7 14.0 21.0 
243 - - - - 7.7 40.6 51.7 - - - - - 0.7 2.5 8.0 19.1 21.9 162 31.7 
251 - - - - 5.2 32.1 62.7 - - - - - 0.3 0.8 54 13.5 18.7 16.9 44.4 
187 - - - - - 78.5 16.7 4.9 - - - - 568 18.6 7.0 12.0 3.1 1.0 1.4 
196 - - - - - 743 18.8 6.9 - - - - 403 253 8.0 14.6 6.7 2.2 3.0 
205 - - - - - 649 264 8.7 - - - - 256 27.7 106 17.2 11.6 3.8 3.6 
7 223 - - - = - 449 36.0 19.1 - - - - 56 13.8 13.3 23.7 20.3 11.1 12.3 
242 - - - - - 328 36.7 30.5 - - - - 0.9 2.6 89 19.5 21.8 16.3 30.0 
251 - - - - - 238 36.1 40.1 - - - - 0.3 0.9 49 135 186 17.1 44.7 
260 - - - - - 136 39.9 46.5 - - - - 0.1 0.2 1.5 6.1 156 14.0 62.5 
190 - - - - - - 542 22.2 23.7 - - - 52.2 20.7 7.1 12.6 4.1 1.4 2.0 
200 - - - - - - 546 226 22.8 - - - 33.7 27.3 9.3 15.2 8.6 2.7 3.2 
209 - - - - - - 600 22.1 17.8 - - - 188 26.0 116 18.2 15.0 5.6 4.7 
8 229 - - - - - - 566 256 17.9 - - - 3.7 90 12.5 23.9 21.3 13.3 16.3 
248 - - - - - - 33.3 25.7 41.0 - - - 0.4 1.4 63 159 198 17.0 39.2 
258 - - - - - - 20.7 188 60.6 - - - 0.1 0.4 2.2 74 15.7 14.3 59.9 
267 - - - - - - 142 12.5 73.3 - - - - 0.1 0.5 1.6 128 11.3 73.7 
Min. 3.1 4.1 7.3 5.1 3.2 5.5 14.2 49 17.8 0.1 0.1 0.2 0.1 0.1 0.2 0.1 0.1 0.1 0.1 
Max. | 19.6 785 69.5 81.6 456 785 62.7 465 73.3 | 37.1 29.0 37.2 61.9 406 25.7 31.1 227 17.1 73.7 
Mean | 12.6 40.2 355 46.0 203 37.7 359 219 36.7 | 108 118 17.7 183 17.3 10.3 12.9 10.4 6.8 14.3 
SD 6.3 25.4 216 224 129 234 162 11.9 224/101 10.0 13.8 16.1 13.0 7.9 9.7 7.6 6.2 20.9 
Notes: SG refers to simulee grade. Modal exposure rate for each RIT level and grade is shown in red. 
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Table C.2. Item Exposure Rates for On-Grade Simulees—Reading 


Appendix C: Item Exposure Rates 


+1 Grade Item Pool 


All-Grade Item Pool 


True 
SG RIT G1 G2 G3 G4 G5 G6 G7 G8 G9 K G1 G2 G3 G4 G5 G6 G7 G8 G9-10 G11-12 
145 | 51.3 35.8 13.0 - - - - - -— | 21.2 264 36.1 10.0 6.3 - - = = - - 
159 | 506 36.1 13.3 - - - - - -| 200 253 362 11.9 6.6 0.1 - - - - - 
167 | 49.4 344 16.2 - - - - - -| 14.7 241 33.0 17.9 9.4 0.9 - - - - - 
2 183 | 17.5 35.1 47.5 - - - - - - 1.7 7.1 22.3 334 249 9.6 1.0 0.2 - - - 
199 1.6 17.3 81.0 - - - - - - - 0.5 5.8 22.4 29.9 22.0 13.0 4.6 1.7 0.1 0.1 
207 0.3 #129 86.8 - - - - - - - 0.1 1.7 104 228 257 23.2 11.2 4.3 0.5 0.2 
215 - 13.3 86.7 - - - - - - - - 0.6 3.8 11.3 19.3 324 18.9 11.2 2.0 0.6 
164 - 72.7 16.0 11.4 - - - - -/| 192 255 351 13.1 7.0 0.2 - - - - - 
172 - 67.7 186 13.7 - - - - -—/} 122 23.3 31.7 20.2 11.1 1.4 - - - - - 
179 - 54.7 28.3 17.0 - - - - -| 49 14.0 27.2 288 19.2 5.7 0.2 - - - - 
3 194 - 21.0 43.9 35.1 - - - - - 0.1 1.6 114 308 296 16.9 6.7 2.1 0.7 - - 
209 - 46 266 68.8 - - - - - - 0.1 1.4 84 20.00 24.7 25.7 13.4 5.4 0.7 0.3 
216 - 44 17.9 77.7 - - - - - - - 0.5 2.7 96 17.3 33.6 20.1 12.9 2.6 0.8 
224 - 44 16.0 79.7 - - - - - - - 0.2 0.5 4.5 7.6 32.7 26.0 22.2 5.4 0.9 
174 - - 80.5 19.3 0.2 - - - -/ 103 22.2 308 223 12.2 2.2 - - - - - 
181 - - 78.0 20.9 1.1 - - - -| 44 13.4 263 296 19.9 6.0 0.3 0.1 - - - 
189 - - 69.6 25.7 4.7 - - - - 0.8 45 194 342 265 11.7 2.2 0.5 0.2 - - 
4 205 - - 36.3 39.0 24.7 - - - - - 0.3 4.0 180 28.7 242 15.9 6.4 2:2 0.2 0.1 
220 - - 95 326 57.9 - - - - - - 0.5 2.7 90 168 339 204 13.3 2.6 0.9 
228 - - 42 30.0 65.7 - - - - - - 0.2 0.6 4.7 8.3 33.1 256 21.8 4.8 0.9 
236 - - 33 294 67.3 - - - - - - - 0.1 3.5 2.6 27.3 294 28.8 7.7 0.5 
180 - - - 98.1 1.5 0.5 - - -| 47 143 269 29.3 186 6.0 0.3 - - - - 
188 - - - 92.0 5.4 2.6 - - - 1.1 5.2 20.5 33.8 26.3 11.0 1.6 0.4 0.1 - - 
197 - - - 800 11.6 8.4 - - - 0.1 1.8 123 314 299 16.1 6.1 1.7 0.6 - - 
5 214 - - - 39.3 23.6 37.1 - - - - 0.1 1.5 93 21.8 251 245 12.1 4.9 0.6 0.3 
230 - - - 88 14.3 76.9 - - - - - 0.3 1.1 5.3 10.8 340 244 19.1 4.2 0.9 
239 - - - 449 6.5 88.6 - - - - - - 0.2 3.7 3.6 28.8 28.5 27.4 7.1 0.7 
247 - - - 3.4 4.1 92.6 - - - - - - - 27 06 242 286 32.8 10.7 0.3 
183 - - - - 93.5 3.7 2.8 - - - - 5.6 37.1 41.4 13.7 1.5 0.4 0.1 - - 
192 - - - - 86.6 9.7 3.7 - - - - 46 33.0 368 17.6 5.5 ule 4 0.5 0.1 0.1 
200 - - - - 73.1 21.4 5.5 - - - - 3.0 23.4 32.0 21.5 12.3 4.8 2.0 0.8 0.3 
6 218 - - - - 394 41.7 18.9 - - - - 0.6 49 12.7 17. 246 15.1 11.6 9.0 4.0 
234 - - - - 7.2 464 46.4 - - - - - 0.2 1.2 2.3 10.7 12.3 22.1 30.6 20.6 
243 - - - - 1.8 385 59.8 - - - - - - 0.2 0.7 5.2 9.0 19.4 37.1 28.3 
251 - - - - 03 33.4 66.4 - - - - - - - 0.2 2.4 5.6 13.9 40.0 37.9 
187 - - - - - 861 12.4 1.6 - - - 5.3 35.9 39.3 15.3 2.8 1.0 0.2 0.1 - 
196 - - - - - 83.2 13.2 3.6 - - - 3.7 29.1 344 19.5 8.6 3.1 1.2 0.4 0.1 
205 - - - - - 784 13.5 8.1 - - - 1.8 178 28.9 229 16.4 6.9 3.2 1.6 0.5 
7 223 - - - - - 550 21.3 23.7 - - - 0.4 2.6 8.2 13.7 234 16.7 15.5 13.0 6.6 
242 - - - - - 236 265 49.9 - - - - 0.1 0.5 1.3 8.0 109 20.8 34.0 24.5 
251 - - - - - 156 27.8 56.6 - - - - - 0.1 0.3 3.7 7.5 166 39.2 32.6 
260 - - - - - 134 28.7 57.9 - - - - - - 0.1 1.3 3.7 11.9 41.3 41.8 
190 - - - - - - 80.7 15.7 3.7 - - 46 32.7 37.1 17.4 5.5 1.9 0.6 0.1 0.1 
200 - - - - - - 788 17.0 4.2 - - 29 23.8 325 21.2 12.4 4.3 2.0 0.8 0.3 
209 - - - - - - 73.2 20.5 6.4 - - 1.3. 122 23.6 23.0 20.6 10.0 5.2 3.1 1.0 
8 229 - - - - - - 485 348 16.7 - - 0.2 1.3 4.5 89 20.1 165 19.4 19.1 10.0 
248 - - - - - - 27.3 449 27.8 - - - - 0.3 0.7 5.5 9.1 19.2 36.9 28.3 
258 - - - - - - 241 44.3 31.6 - - - - 0.1 0.2 2.5 5.5 14.2 40.3 37.3 
267 - - - - - - 231 43.1 33.8 - - - - - 0.1 0.7 1.8 10.3 42.0 45.2 
Min. 0.3 4.4 3.3 3.4 0.2 0.5 2.8 1.6 3.7 0.1 0.1 0.2 0.1 0.1 0.1 0.2 0.1 0.1 0.1 0.1 
Max. | 51.3. 72.7 86.8 981 93.5 926 80.7 57.9 33.8 | 21.2 264 36.2 37.1 414 25.7 340 29.4 32.8 42.0 45.2 
Mean | 28.5 296 37.8 394 281 408 334 301 17.7 82 11.0 11.3 166 165 10.7 138 103 10.8 12.9 9.9 
SD 6.3 25.4 216 224 #129 234 #162 11.9 224/101 #100 138 16.1 13.0 7.9 9.7 7.6 6.2 20.9 - 
Notes: SG refers to simulee grade. Modal exposure rate for each RIT level and grade is shown in red. 
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Appendix C: Item Exposure Rates 


Table C.3. Item Exposure Rates for On-Grade Simulees—Language Usage 


+1 Grade Item Pool 


All-Grade Item Pool 


True 
SG RIT G1 G2 G3 G4 G5 G6 G7 G8 G9 K G1 G2 G3 G4 G5 G6 G7 G8 G9-10 G11-12 
145 | 51.3 35.8 13.0 - - - - - -— | 21.2 264 36.1 10.0 6.3 - - = = - - 
159 | 506 36.1 13.3 - - - - - -| 200 253 362 11.9 6.6 0.1 - - - - - 
167 | 49.4 344 16.2 - - - - - -| 14.7 241 33.0 17.9 9.4 0.9 - - - - - 
2 183 | 17.5 35.1 47.5 - - - - - - 1.7 7.1 22.3 334 249 9.6 1.0 0.2 - - - 
199 1.6 17.3 81.0 - - - - - - - 0.5 5.8 22.4 29.9 22.0 13.0 4.6 1.7 0.1 0.1 
207 0.3 #129 86.8 - - - - - - - 0.1 1.7 104 228 257 23.2 11.2 4.3 0.5 0.2 
215 - 13.3 86.7 - - - - - - - - 0.6 3.8 11.3 19.3 324 18.9 11.2 2.0 0.6 
164 - 72.7 16.0 11.4 - - - - -/| 192 255 351 13.1 7.0 0.2 - - - - - 
172 - 67.7 186 13.7 - - - - -—/} 122 23.3 31.7 20.2 11.1 1.4 - - - - - 
179 - 54.7 28.3 17.0 - - - - -| 49 14.0 27.2 288 19.2 5.7 0.2 - - - - 
3 194 - 21.0 43.9 35.1 - - - - - 0.1 1.6 114 308 296 16.9 6.7 2.1 0.7 - - 
209 - 46 266 68.8 - - - - - - 0.1 1.4 84 20.0 24.7 25.7 13.4 5.4 0.7 0.3 
216 - 44 17.9 77.7 - - - - - - - 0.5 2.7 96 17.3 33.6 20.1 12.9 2.6 0.8 
224 - 44 16.0 79.7 - - - - - - - 0.2 0.5 4.5 7.6 32.7 26.0 22.2 5.4 0.9 
174 - - 80.5 19.3 0.2 - - - -/ 103 22.2 308 223 12.2 2.2 - - - - - 
181 - - 78.0 20.9 1.1 - - - -| 44 13.4 263 296 19.9 6.0 0.3 0.1 - - - 
189 - - 69.6 25.7 4.7 - - - - 0.8 45 194 342 265 11.7 2.2 0.5 0.2 - - 
4 205 - - 36.3 39.0 24.7 - - - - - 0.3 4.0 180 28.7 242 15.9 6.4 2:2 0.2 0.1 
220 - - 95 326 57.9 - - - - - - 0.5 2.7 90 168 339 204 13.3 2.6 0.9 
228 - - 42 30.0 65.7 - - - - - - 0.2 0.6 4.7 8.3 33.1 256 21.8 4.8 0.9 
236 - - 33 294 67.3 - - - - - - - 0.1 3.5 2.6 27.3 294 28.8 7.7 0.5 
180 - - - 98.1 1.5 0.5 - - -| 47 143 269 29.3 186 6.0 0.3 - - - - 
188 - - - 92.0 5.4 2.6 - - - 1.1 5.2 20.5 33.8 26.3 11.0 1.6 0.4 0.1 - - 
197 - - - 800 11.6 8.4 - - - 0.1 1.8 123 314 299 16.1 6.1 1.7 0.6 - - 
5 214 - - - 39.3 23.6 37.1 - - - - 0.1 1.5 93 21.8 251 245 12.1 4.9 0.6 0.3 
230 - - - 88 14.3 76.9 - - - - - 0.3 1.1 5.3 10.8 340 244 19.1 4.2 0.9 
239 - - - 449 6.5 88.6 - - - - - - 0.2 3.7 3.6 28.8 28.5 27.4 7.1 0.7 
247 - - - 3.4 4.1 92.6 - - - - - - - 27 06 242 286 32.8 10.7 0.3 
183 - - - - 93.5 3.7 2.8 - - - - 5.6 37.1 41.4 13.7 1.5 0.4 0.1 - - 
192 - - - - 86.6 9.7 3.7 - - - - 46 33.0 368 17.6 5.5 ule 4 0.5 0.1 0.1 
200 - - - - 73.1 21.4 5.5 - - - - 3.0 23.4 32.0 21.5 12.3 4.8 2.0 0.8 0.3 
6 218 - - - - 394 41.7 18.9 - - - - 0.6 49 12.7 17.5 246 15.1 11.6 9.0 4.0 
234 - - - - 7.2 464 46.4 - - - - - 0.2 1.2 2.3 10.7 12.3 22.1 30.6 20.6 
243 - - - - 1.8 385 59.8 - - - - - - 0.2 0.7 5.2 9.0 19.4 37.1 28.3 
251 - - - - 03 33.4 66.4 - - - - - - - 0.2 2.4 5.6 13.9 40.0 37.9 
187 - - - - - 861 12.4 1.6 - - - 5.3 35.9 39.3 15.3 2.8 1.0 0.2 0.1 - 
196 - - - - - 83.2 13.2 3.6 - - - 3.7 29.1 344 19.5 8.6 3.1 1.2 0.4 0.1 
205 - - - - - 784 13.5 8.1 - - - 1.8 178 28.9 229 16.4 6.9 3.2 1.6 0.5 
7 223 - - - - - 550 21.3 23.7 - - - 0.4 2.6 8.2 13.7 234 16.7 15.5 13.0 6.6 
242 - - - - - 236 265 49.9 - - - - 0.1 0.5 1.3 8.0 10.9 20.8 34.0 24.5 
251 - - - - - 156 27.8 56.6 - - - - - 0.1 0.3 3.7 7.5 166 39.2 32.6 
260 - - - - - 134 28.7 57.9 - - - - - - 0.1 1.3 3.7 11.9 41.3 41.8 
190 - - - - - - 80.7 15.7 3.7 - - 46 32.7 37.1 17.4 5.5 1.9 0.6 0.1 0.1 
200 - - - - - - 788 17.0 4.2 - - 29 238 325 21.2 12.4 4.3 2.0 0.8 0.3 
209 - - - - - - 73.2 20.5 6.4 - - 1.3. 122 23.6 23.0 20.6 10.0 5.2 3.1 1.0 
8 229 - - - - - - 485 348 16.7 - - 0.2 1.3 4.5 89 20.1 165 19.4 19.1 10.0 
248 - - - - - - 27.3 449 27.8 - - - - 0.3 0.7 5.5 9.1 19.2 36.9 28.3 
258 - - - - - - 241 443 31.6 - - - - 0.1 0.2 2.5 5.5 14.2 40.3 37.3 
267 - - - - - - 231 43.1 33.8 - - - - - 0.1 0.7 1.8 10.3 42.0 45.2 
Min. 0.3 4.4 3.3 3.4 0.2 0.5 2.8 1.6 3.7 0.1 0.1 0.2 0.1 0.1 0.1 0.2 0.1 0.1 0.1 0.1 
Max. | 51.3. 72.7 86.8 981 93.5 926 80.7 57.9 33.8 | 21.2 264 362 37.1 414 25.7 340 29.4 32.8 42.0 45.2 
Mean | 28.5 296 37.8 394 281 408 334 301 17.7 82 11.00 11.3 166 165 10.7 138 103 108 12.9 9.9 
SD 6.3 254 216 224 #129 234 162 11.9 224/101 #100 138 16.1 13.0 7.9 9.7 7.6 6.2 20.9 - 
Notes: SG refers to simulee grade. Modal exposure rate for each RIT level and grade is shown in red. 
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Appendix C: Item Exposure Rates 


Table C.4. Item Exposure Rates for Off-Grade Simulees—Mathematics 


+1 Grade Item Pool 


All-Grade Item Pool 


True 
SG RIT G1 G2 G3 G4 G5 G6 G7 G8 G9 K G1 G2 G3 G4 G5 G6 G7 G8 G9 
133 | 18.8 76.2 5.0 - - - - - — | 55.0 9.6 32.5 2.9 - - - - - - 
138 | 19.1 76.0 4.9 - - - - - -| 530 10.9 33.3 2.8 0.1 - - = - = 
2 142 | 19.1 75.8 5.1 - - - - - -| 491 13.3 34.7 2.8 0.1 - - = - - 
216 2.9 27.3 69.8 - - - - - - 0.1 1.1 4.3 13.8 37.8 20.3 17.6 4.1 0.5 0.4 
220 24 25.8 71.8 - - - - - - - 0.5 3.1. 11.1 33.6 22.1 22.7 5.4 0.9 0.6 
224 24 240 73.6 - - - - - - - 0.2 2.2 9.3 29.2 23.5 26.0 7.0 1.7 0.9 
151 - 92.1 7.0 0.9 - - - - -— | 37.1 21.1 37.2 4.1 0.5 - - - - = 
159 - 84.7 12.1 3.2 - - - - -—| 275 274 36.9 6.5 1.6 0.2 - - - = 
3 162 - 81.1 14.4 4.5 - - - - -—| 247 284 36.5 7.9 2.1 0.3 0.1 - - = 
228 - 33 23.5 73.3 - - - - - - 0.1 1.4 7.8 24.7 245 28.1 8.9 3.2 1.5 
232 - 20 22.3 75.7 - - - - - - - 0.8 6.0 21.6 25.1 286 10.9 4.9 2.0 
236 - 1.3. 22.3 76.4 - - - - - - - 0.4 45 188 253 29.0 13.0 6.1 2.9 
164 - - 743 23.4 2.3 - - - -—| 224 29.0 36.3 9.1 2.7 0.5 - - - = 
168 - - 726 25.0 2:5 - - - -/| 189 285 360 11.6 4.2 0.7 0.1 - - - 
4 172 - - 706 26.3 3.1 - - - -/|165 255 358 15.9 5.0 0.9 0.3 - - - 
239 - - 60 46.2 47.8 - - - - - - 0.2 3.8 164 256 29.7 14.3 6.4 3.6 
243 - - 45 456 49.9 - - - - - - 0.1 2.7 12.8 25.4 31.0 17.0 6.4 4.6 
247 - - 40 45.5 50.6 - - - - - - - 20 106 251 31.5 19.2 6.5 5.0 
174 - - - 844 11.2 4.4 - - -| 15.1 23.9 344 18.7 6.1 1.3 0.4 - - - 
176 - - - 83.6 11.6 4.8 - - - | 146 21.7 33.7 20.8 7.1 1.8 0.4 - - - 
5 179 - - - 81.8 12.6 5.6 - - -/| 142 184 31.0 249 8.7 2.3 0.5 - - - 
248 - - - 33.0 40.5 26.5 - - - - - - 1.8 103 25.2 31.2 20.2 6.6 4.7 
249 - - - 32.4 40.6 27.0 - - - - - - 1.6 101 25.2 309 20.9 6.6 4.6 
251 - - - 32.7 40.2 27.1 - - - - - - 1.2 98 249 305 22.4 6.8 4.5 
180 - - - - 205 648 14.7 - - - - - 646 16.1 6.7 10.3 1.7 0.4 0.3 
181 - - - - 207 646 14.7 - - - - - 638 16.3 6.6 10.6 2.1 0.3 0.3 
6 182 - - - - 206 644 15.1 - - - - - 63.1 16.7 6.6 10.6 2.2 0.4 0.5 
254 - - - - 43 28.5 67.2 - - - - - 0.3 0.5 3.8 109 17.5 16.0 51.0 
257 - - - - 3.6 245 71.9 - - - - - 0.2 0.3 2.6 8.3 16.3 15.4 57.0 
260 - - - - 28 20.7 76.5 - - - - - 0.1 0.2 1.5 5.9 153 13.9 63.1 
183 - - - - - 79.1 16.2 4.6 - - - - 61.9 17.0 6.8 10.8 2.4 0.5 0.6 
184 - - - - - 79.1 16.2 4.7 - - - - 609 17.2 7.0 10.7 2.6 0.8 0.9 
7 185 - - - - - 788 16.4 4.9 - - - - 59.9 18.0 6.9 11.0 2.6 0.8 0.9 
262 - - - - - 11.7 40.2 48.1 - - - - 0.1 0.2 1.2 44 148 13.1 66.3 
264 - - - - - 9.7 41.2 49.0 - - - - - 0.2 0.9 3.2 143 124 69.0 
267 - - - - - 7.8 42.0 50.3 - - - - - 0.2 0.4 1.8 125 11.1 73.9 
187 - - - - - - 548 21.7 23.5 - - - 568 18.6 7.0 12.0 3.1 1.0 1.4 
189 - - - - - - 545 21.9 23.6 - - - 53.6 20.1 6.8 12.6 3.6 1.3 2.0 
3 190 - - - - - - 542 22.1 23.7 - - - 52.2 20.2 7.0 12.9 4.2 1.4 2.1 
268 - - - - - - 13.9 12.3 73.7 - - - - 0.1 0.4 1.6 11.9 10.8 75.3 
269 - - - - - - 12.7 11.6 75.7 - - - - - 0.3 1.1 11.5 105 76.6 
270 - - - - - - 124 11.5 76.1 - - - - - 0.3 1.14. 10.9 10.0 77.7 
Min. 2.4 il8) 4.0 0.9 28} 44 12.4 4.6 23.5 0.1 0.1 0.1 0.1 0.1 0.2 0.1 1.7 0.3 0.3 
Max. | 19.1 921 743 844 506 79.1 765 503 76.1 | 550 29.0 37.2 646 378 256 31.5 224 16.0 77.7 
Mean | 10.8 47.5 31.3 441 214 349 353 219 494 | 268 162 215 19.8 11.2 98 13.3 10.4 5.9 21.8 
SD | 9.02 36.34 30.35 28.82 18.38 28.31 23.13 17.62 28.28 |16.93 11.15 16.82 23.75 10.15 10.33 11.60 6.59 5.12 31.03 
Notes: SG refers to simulee grade. Modal exposure rate for each RIT level and grade is shown in red. 
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Table C.5. Item Exposure Rates for Off-Grade Simulees—Reading 


Appendix C: Item Exposure Rates 


+1 Grade Item Pool 


All-Grade Item Pool 


True 
SG RIT G1 G2 G3 G4 G5 G6 G7 G8 G9 K G1 G2 G3 G4 G5 G6 G7 G8 G9-10 G11-12 
134 | 51.8 35.2 13.1 - - - - - -/ 215 268 36.3 9.4 6.0 - - = - - - 
138 | 51.7 35.3 13.0 - - - - - -—| 215 26.7 36.2 9.6 6.0 - - - - - - 
2 142 | 51.4 35.6 13.0 - - - - - -/ 215 265 36.4 9.7 6.0 - - - = - - 
218 - 13.1 86.9 - - - - - - - - 0.5 2.7 9.0 16.7 345 205 12.6 2.5 1.0 
223 - 13.2 86.8 = - - - - - - - 0.3 1.3 5.9 11.6 34.7 23.2 18.1 4.1 0.9 
227 - 13.0 87.0 - - - - - - - - 0.1 0.6 44 7.5 32.5 26.2 22.5 5.4 0.9 
145 - 73.6 16.8 9.6 - - - - -—| 21.2 264 36.1 10.0 6.3 - - = - - - 
150 - 73.5 16.5 10.1 - - - - -—| 20.7 26.0 365 10.6 6.2 - - - - - - 
3 154 - 72.6 16.7 10.7 - - - - -—| 201 25.2 363 11.6 6.8 0.1 - - - - - 
229 - 43 15.6 80.1 - - - - - - - 0.1 0.3 4.3 5.8 31.5 26.7 24.4 6.1 0.8 
232 - 43 15.5 80.2 - - - - - - - - 0.2 3.8 3.8 29.5 28.2 26.8 72 0.6 
234 - 44 15.4 80.3 - - - - - - - - 0.1 3.4 2.5 27.4 29.2 29.0 7.9 0.6 
157 - - 814 18.6 - - - - -/| 192 255 35.1 13.1 7.0 0.2 - - - - - 
161 - - 81.3 18.6 - - - - -| 16.7 24.7 33.5 16.3 8.3 0.6 - - - - - 
4 165 - - 809 18.9 0.2 - - - -/ 128 23.14 321 199 10.6 1.5 0.1 - - - - 
236 - - 32 29.3 67.4 - - - - - - - 0.1 3.2 1.8 261 29.3 30.1 9.0 0.5 
238 - - 32 29.1 67.8 - - - - - - - - 3.0 1.00 253 28.7 31.7 9.9 0.4 
241 - - 32 29.2 67.6 - - - - - - - - 27 05 242 28.2 33.3 10.9 0.2 
167 - - - 99.6 0.3 0.1 - - -/ 103 22.2 308 223 12.2 2.2 - - - - - 
170 - - - 99.2 0.6 0.3 - - - 7.8 19.0 285 25.9 15.4 3.3 0.1 - - - - 
5 172 - - - 98.6 1.0 0.4 - - - 5.8 17.0 27.9 27.1 17.3 4.7 0.2 - - - - 
242 - - - 3.5 4.0 92.6 - - - - - - - 26 0.3 23.5 27.7 34.5 11.1 0.2 
244 - - - 3.5 4.0 92.5 - - - - - - - 25 0.2 23.3 27.8 345 11.6 0.1 
245 - - - 3.4 4.0 926 - - - - - - - 25 0.2 23.4 27.7 345 11.6 0.1 
174 - - - - 94.9 2.6 2.5 - - - - 54 39.7 43.5 10.7 0.5 0.1 = - - 
176 - - - - 94.7 2.7 2.6 - - - - 56 39.2 428 11.6 0.7 0.2 = - - 
6 179 - - - - 93.9 3.4 2.7 - - - - 5.5 37.6 421 12.9 1.4 0.4 0.1 - - 
246 - - - - 0.2 33.5 66.3 - - - - - - 0.1 0.2 2.0 5.2 13.4 40.5 38.6 
248 - - - - 0.1 33.4 66.5 - - - - - - - 0.1 1.5 43 12.3 41.0 40.8 
249 - - - - 0.1 33.3 66.6 - - - - - - - 0.1 1.3 3.7 11.9 41.3 41.7 
180 - - - - - 865 12.3 1.3 - - - 5.6 37.1 41.4 13.7 1.5 0.4 0.1 - - 
182 - - - - - 863 12.4 1.3 - - - 5.9 36.2 403 14.7 2.1 0.7 0.2 - - 
7 183 - - - - - 86.3 12.3 1.4 - - - 5.6 36.0 394 15.3 2.7 0.8 0.2 - - 
250 - - - - - 136 28.2 58.2 - - - - - - 0.1 1.2 3.1 11.1 41.1 43.4 
252 - - - - - 135 284 58.1 - - - - - - 0.1 0.9 2.1 10.7 41.3 44.9 
253 - - - - - 13.9 28.1 58.0 - - - - - - 0.1 0.6 18 10.5 41.6 45.5 
184 - - - - - - 806 15.6 3.8 - - 5.3 35.9 39.3 15.3 2.8 1.0 0.2 0.1 - 
186 - - - - - - 805 15.7 3.8 - - 5.3 34.1 382 16.6 4.0 1.4 0.3 0.1 0.1 
3 187 - - - - - - 806 15.7 3.7 - - 5.0 33.7 37.5 17.0 4.8 1.4 0.5 0.1 0.1 
254 - - - - - - 231 43.4 33.5 - - - - - - 0.5 1.5 102 42.1 45.7 
255 - - - - - - 232 43.6 33.3 - - - - - - 0.4 1.5 10.3 42.2 45.6 
256 - - = - - - 231 43.4 33.5 - - - - - - 0.3 1.3. 10.4 42.7 45.3 
Min. | 51.4 4.3 3.2 3.4 0.1 0.1 2.5 1.3 3.7 5.8 17.0 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 
Max. | 51.8 73.6 87.0 996 949 926 806 582 33.5 | 215 268 365 39.7 435 17.0 34.7 29.3 345 42.7 45.7 
Mean | 51.6 31.55 36.1 401 31.3 382 356 296 186 | 166 241 182 186 15.3 5.7 11.1 11.8 15.5 19.6 17.3 
SD 0.2 27.8 35.2 374 40.7 38.9 291 23.3 16.2 5.8 3.2 155 14.7 15.8 6.3 13.3. 12.7 12.4 17.7 21.5 
Notes: SG refers to simulee grade. Modal exposure rate for each RIT level and grade is shown in red. 
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Appendix C: Item Exposure Rates 


Table C.6. Item Exposure Rates for Off-Grade Simulees—Language Usage 


+1 Grade Item Pool 


All-Grade Item Pool 


True 
SG RIT G1 G2 G3 G4 G5 G6 G7 G8 G9 K G1 G2 G3 G4 G5 G6 G7 G8 G9-10 G11-12 
134 | 44.7 25.1 30.2 - - - - - -/| 119 341 168 29.7 7.5 - - - = - - 
138 | 44.5 25.1 30.3 - - - - - -/|119 33.9 168 29.8 7.6 - - - - - - 
2 142 | 44.4 25.2 30.4 - - - - - -/| 120 33.8 16.7 29.7 7.8 - - - - - - 
216 1.2 6.9 91.9 - - - - - - - 0.4 1.4 90 18.0 142 144 165 108 10.5 4.7 
220 0.6 5.3 94.1 = - - - - - - 0.1 0.7 65 126 11.3 148 19.2 13.2 15.1 6.7 
224 0.3 46 95.2 - - - - - - - - 0.3 4.8 8.4 9.0 13.7 205 152 19.5 8.7 
155 - 650.0 31.8 18.2 - - - - -/| 125 32.0 176 29.1 8.6 0.1 - - - - - 
157 - 496 328 17.6 - - - - -/| 124 308 183 29.3 9.0 0.1 - 0.1 - - - 
3 159 - 48.7 344 17.0 - - - - -—| 121 30.7 187 29.6 8.7 0.1 - - - - - 
226 - 07 304 68.8 - - - - - - - 0.2 3.9 7.2 7.6 13.2 20.8 15.7 21.6 9.9 
228 - 04 31.6 68.0 - - - - - - - 0.1 3.7 5.9 74 12.4 21.3 15.9 22.7 10.6 
231 - 02 326 67.1 - - - - - - - - 3.4 4.9 69 110 214 162 244 11.9 
160 - - 72.8 25.5 1.7 - - - -/| 118 308 19.1 29.2 8.8 0.2 - 0.1 - - - 
165 - - 75.5 22.9 1.6 - - - - 9.7 28.9 206 30.4 9.5 0.4 0.1 0.2 - - - 
4 169 - - 77.8 19.9 2.3 - - - - 8.2 27.6 208 31.1 10.9 0.8 0.2 0.4 - - - 
232 - - 201 285 51.4 - - - - - - - 3.3 4.8 66 10.2 21.5 165 246 12.4 
234 - - 204 27.7 52.0 - - - - - - - 3.2 4.3 6.7 98 20.1 17.5 25.7 12.8 
236 - - 200 27.8 52.2 - - - - - - - 28 3.7 6.3 93 19.4 17.6 26.5 14.3 
170 - - - 91.4 5.6 3.0 - - - 7.8 27.6 205 31.2 11.2 1.0 0.3 0.5 - - - 
173 - - - 90.3 6.7 3.0 - - - 64 264 203 314 13.3 1.5 0.3 0.5 - - - 
5 176 - - - 87.6 9.1 3.3 - - - 5.2 26.0 19.7 31.0 14.4 2:3 0.7 0.6 - - - 
237 - - - 15.0 23.9 61.0 - - - - - - 2.8 3.5 6.2 9.1 17.9 18.3 27.5 14.8 
239 - - - 143 23.7 62.0 - - - - - - 25 3.3 6.0 8.7 16.4 20.1 27.9 15.1 
241 - - - 143 23.3 62.4 - - - - - - 24 2.8 5.7 84 15.0 21.0 29.3 15.8 
177 - - - - 781 11.3 10.6 - - 5.0 25.0 19.5 314 14.9 3.0 0.7 0.5 = - - 
179 - - - - 788 11.1 10.2 - - 3.9 23.4 194 31.2 164 4.1 1.0 0.6 - - - 
6 180 - - - - 79.0 11.1 9.9 - - 3.7 22.8 19.2 31.2 16.5 4.6 1.2 0.7 0.1 - - 
242 - - - - 206 36.1 43.3 - - - - - 1.9 2.7 5.5 84 141 214 29.5 16.5 
243 - - - - 206 36.2 43.2 - - - - - 1.8 2.6 5.2 8.0 13.4 21.4 30.5 17.1 
244 - - - - 205 36.2 43.3 - - - - - 1.6 2.3 5.4 8.2 12.5 21.9 30.7 17.4 
181 - - - - - 744 214 4.2 - 3.4 21.7 194 31.6 16.5 5.4 1.2 0.7 0.1 - - 
183 - - - - - 746 21.0 4.4 -/} 29 20.2 192 309 17.6 6.7 1.6 0.7 0.1 - - 
7 184 - - - - - 746 20.9 4.5 -/} 23 195 188 30.3 186 7.6 1.9 0.9 0.1 - - 
245 - - - - - 262 278 46.1 - - - - 1.4 23 5.1 8.3 12.3 21.9 31.2 17.5 
247 - - - - - 266 27.3 46.2 - - - - 1.0 2:3 5.1 8.1 11.5 21.6 32.1 18.3 
248 - - - - - 265 268 46.7 - - - - 1.1 2.1 4.9 7.8 611.1 21.5 33.2 18.4 
185 - - - - - - 804 13.8 5.8 2.3 179 179 304 19.9 8.2 2.2 1.0 0.2 - - 
186 - - - - - - 806 13.7 5.7 2.3 169 17.8 30.0 204 9.1 2.3 1.0 0.2 - 0.1 
3 188 - - - - - - 805 13.7 5.9 1.7 146 16.4 302 21.6 10.7 3.0 1.4 0.5 0.1 0.1 
249 - - - - - - 302 41.8 27.9 - - - 0.9 2.1 5.0 7.8 109 21.4 33.2 18.7 
250 - - - - - - 302 41.9 27.9 - - - 0.8 2.1 5.1 7.7 106 21.4 33.6 18.7 
251 - - = - - - 302 41.8 27.9 - - - 0.8 2.1 5 7.7 10.5 21.3 33.7 18.9 
Min. 0.3 0.2 20.0 14.3 1.6 3.0 9.9 4.2 5.7 1.7 0.1 0.1 0.8 2.1 0.1 0.1 0.1 0.1 0.1 0.1 
Max. | 44.7. 50.00 95.2 914 79.0 746 806 46.7 27.9 | 125 341 208 316 216 142 148 21.5 21.9 33.7 18.9 
Mean | 22.6 20.2 47.3 40.1 306 355 354 266 16.9 7.1 23.7 15.2 16.6 9.0 5.3 6.4 94 140 25.6 13.0 
SD | 24.0 201 28.0 29.3 27.5 264 23.1 187 12.1 4.1 9.3 74 14.0 6.1 3.3 4.7 8.3 8.6 8.3 5.7 
Notes: SG refers to simulee grade. Modal exposure rate for each RIT level and grade is shown in red. 
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Appendix D: Pool Utilization 


Table D.1. Pool Utilization for On-Grade Simulees—Mathematics 


Appendix D: Pool Utilization 


On-Grade Item Pool +1Grade Item Pool All-Grade Item Pool 

Total Items Used Total Items Used Total Items Used 

Grade | True RIT #ltems N % #items N % #ltems N % 
147 445 98 22.0 1,621 201 12.4 5,536 235 4.2 
155 445 164 36.9 1,621 395 24.4 5,536 464 8.4 
164 445 283 63.6 1,621 726 44.8 5,536 966 17.4 

2 180 445 442 99.3 1,621 1,385 85.4 5,536 2,218 40.1 
197 445 388 87.2 1,621 1,292 79.7 5,536 3,539 63.9 
206 445 291 65.4 1,621 917 56.6 5,536 3,720 67.2 
214 445 173 38.9 1,621 611 37.7 5,536 3,388 61.2 
160 922 248 26.9 1,771 391 22.1 5,536 719 13.0 
168 922 405 43.9 1,771 652 36.8 5,536 1,196 21.6 
176 922 637 69.1 1,771 960 54.2 5,536 1,828 33.0 
3 192 922 899 97.5 1,771 1,613 91.1 5,536 3,291 59.4 
208 922 674 73.1 1,771 1,294 73.1 5,536 3,706 66.9 
216 922 463 50.2 1,771 926 52.3 5,536 3,233 58.4 
224 922 265 28.7 1,771 526 29.7 5,536 2,370 42.8 
170 1,004 298 29.7 1,910 578 30.3 5,536 1,344 24.3 
177 1,004 467 46.5 1,910 907 47.5 5,536 1,892 34.2 

185 1,004 738 73.5 1,910 1,336 69.9 5,536 2,717 49.1 
4 201 1,004 979 97.5 1,910 1,705 89.3 5,536 3,697 66.8 
216 1,004 736 73.3 1,910 1,150 60.2 5,536 3,246 58.6 
224 1,004 432 43.0 1,910 686 35.9 5,536 2,383 43.0 
231 1,004 238 23.7 1,910 415 21.7 5,536 1,657 29.9 
177 681 258 37.9 1,730 575 33.2 5,536 1,904 34.4 
184 681 420 61.7 1,730 900 52.0 5,536 2,584 46.7 
192 681 594 87.2 1,730 1,322 76.4 5,536 3,266 59.0 
5 207 681 680 99.9 1,730 1,598 92.4 5,536 3,729 67.4 
222 681 478 70.2 1,730 980 56.6 5,536 2,609 47.1 
229 681 297 43.6 1,730 642 37.1 5,536 1,855 33.5 
236 681 189 27.8 1,730 384 22.2 5,536 1,196 21.6 
181 512 130 25.4 1,284 423 32.9 5,536 2,309 41.7 
189 512 263 51.4 1,284 675 52.6 5,536 2,988 54.0 
196 512 406 79.3 1,284 958 74.6 5,536 3,558 64.3 
6 211 512 509 99.4 1,284 1,203 93.7 5,536 3,636 65.7 
226 512 448 87.5 1,284 846 65.9 5,536 2,173 39.3 
233 512 307 60.0 1,284 571 44.5 5,536 1,486 26.8 
241 512 179 35.0 1,284 338 26.3 5,536 858 15.5 
185 509 133 26.1 1,005 246 24.5 5,536 2,692 48.6 

193 509 267 52.5 1,005 473 47.1 5,536 3,384 61.1 
200 509 374 73.5 1,005 718 71.4 5,536 3,676 66.4 
7 215 509 504 99.0 1,005 978 97.3 5,536 3,362 60.7 
230 509 447 87.8 1,005 785 78.1 5,536 1,727 31.2 
237 509 332 65.2 1,005 533 53.0 5,536 1,125 20.3 
244 509 205 40.3 1,005 335 33.3 5,536 733 13.2 
189 366 120 32.8 861 227 26.4 5,536 3,035 54.8 
196 366 209 57.1 861 435 50.5 5,536 3,527 63.7 
204 366 306 83.6 861 660 76.7 5,536 3,702 66.9 
8 218 366 363 99.2 861 852 99.0 5,536 3,011 54.4 
233 366 342 93.4 861 692 80.4 5,536 1,479 26.7 
241 366 248 67.8 861 460 53.4 5,536 861 15.6 
248 366 159 43.4 861 296 34.4 5,536 509 9.2 
Min. - - 22.0 - - 12.4 - - 4.2 
Max. = - 99.9 - - 99.0 - - 67.4 
Mean = - 60.8 - - 53.9 - - 42.3 
SD - - 25.2 - - 23.9 - - 19.6 
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Table D.2. Pool Utilization for On-Grade Simulees—Reading 


Appendix D: Pool Utilization 


On-Grade Item Pool +1Grade Item Pool All-Grade Item Pool 

Total Items Used Total Items Used Total Items Used 

Grade True RIT #iltems N % #items N % #iltems N % 
145 584 126 21.6 1,324 157 11.9 5,727 218 3.8 
154 584 274 46.9 1,324 329 24.8 5,727 536 9.4 
163 584 440 75.3 1,324 601 45.4 5,727 1,073 18.7 

2 180 584 578 99.0 1,324 1,165 88.0 5,727 2,471 43.1 
198 584 351 60.1 1,324 932 70.4 5,727 3,740 65.3 
207 584 193 33.0 1,324 649 49.0 5,727 3,694 64.5 
216 584 85 14.6 1,324 345 26.1 5,727 2,786 48.6 
157 1,072 231 21.5 2,012 398 19.8 5,727 722 12.6 
165 1,072 446 41.6 2,012 690 34.3 5,727 1,231 21.5 
174 1,072 714 66.6 2,012 1,067 53.0 5,727 1,867 32.6 
3 192 1,072 1,035 96.5 2,012 1,754 87.2 5,727 3,395 59.3 
209 1,072 567 52.9 2,012 1,124 55.9 5,727 3,612 63.1 
218 1,072 298 27.8 2,012 645 32.1 5,727 2,556 44.6 
227 1,072 125 11.7 2,012 278 13.8 5,727 1,468 25.6 
167 1,216 339 27.9 2,271 602 26.5 5,727 1,348 23.5 
175 1,216 523 43.0 2,271 952 41.9 5,727 1,973 34.5 
184 1,216 883 72.6 2,271 1,505 66.3 5,727 2,874 50.2 
4 201 1,216 1,117 91.9 2,271 1,902 83.8 5,727 3,856 67.3 
218 1,216 563 46.3 2,271 995 43.8 5,727 2,476 43.2 
226 1,216 284 23.4 2,271 504 22.2 5,727 1,579 27.6 
234 1,216 124 10.2 2,271 224 9.9 5,727 784 13.7 
174 969 249 25.7 2,441 551 22.6 5,727 1,831 32.0 
183 969 471 48.6 2,441 991 40.6 5,727 2,719 47.5 
181 969 767 79.2 2,441 1,552 63.6 5,727 3,317 57.9 
5 208 969 925 95.5 2,441 2,144 87.8 5,727 3,658 63.9 
224 969 494 51.0 2,441 1,143 46.8 5,727 1,810 31.6 
232 969 266 27.5 2,441 565 23.1 5,727 963 16.8 
241 969 88 9.1 2,441 247 10.1 5,727 423 74 

180 1,036 199 19.2 2,106 457 21.7 6,472 1,495 23.1 
188 1,036 390 37.6 2,106 840 39.9 6,472 2,432 37.6 
196 1,036 688 66.4 2,106 1,381 65.6 6,472 3,402 52.6 
6 212 1,036 1,029 99.3 2,106 1,958 93.0 6,472 4,428 68.4 
229 1,036 645 62.3 2,106 971 46.1 6,472 2,703 41.8 

237 1,036 349 33.7 2,106 520 24.7 6,472 1692 26.1 
245 1,036 167 16.1 2,106 249 11.8 6,472 931 14.4 
184 697 166 23.8 1,893 339 17.9 6,472 1,906 29.4 
192 697 307 44.0 1,893 623 32.9 6,472 2,907 44.9 
200 697 514 73.7 1,893 1,131 59.7 6,472 3,853 59.5 
7 216 697 695 99.7 1,893 1,807 95.5 6,472 4,202 64.9 
233 697 452 64.8 1,893 991 52.4 6,472 2,135 33.0 
241 697 251 36.0 1,893 537 28.4 6,472 1,217 18.8 
249 697 131 18.8 1,893 281 14.8 6,472 657 10.2 
188 710 132 18.6 1,382 250 18.1 6,472 2,460 38.0 
196 710 266 37.5 1,382 514 37.2 6,472 3,404 52.6 

204 710 480 67.6 1,382 911 65.9 6,472 4,149 64.1 
8 220 710 707 99.6 1,382 1,356 98.1 6,472 3,866 59.7 
237 710 507 71.4 1,382 747 54.1 6,472 1,704 26.3 
245 710 286 40.3 1,382 429 31.0 6,472 893 13.8 
253 710 144 20.3 1,382 193 14.0 6,472 425 6.6 
Min. - - 9.1 - - 9.9 - - 3.8 
Max. - - 99.7 - - 98.1 - - 68.4 

Mean = - 48.4 - - 43.3 - - 37.1 
SD - - 27.9 - - 25.6 - - 19.5 
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Appendix D: Pool Utilization 


Table D.3. Pool Utilization for On-Grade Simulees—Language Usage 


On-Grade Item Pool +1Grade Item Pool All-Grade Item Pool 

Total Items Used Total Items Used Total Items Used 

Grade True RIT #iltems N % #iltems N % #iltems N % 
147 445 98 22.0 1,621 201 12.4 5,536 235 4.2 
155 445 164 36.9 1,621 395 24.4 5,536 464 8.4 
164 445 283 63.6 1,621 726 44.8 5,536 966 17.4 

2 180 445 442 99.3 1,621 1,385 85.4 5,536 2,218 40.1 
197 445 388 87.2 1,621 1,292 79.7 5,536 3,539 63.9 
206 445 291 65.4 1,621 917 56.6 5,536 3,720 67.2 
214 445 173 38.9 1,621 611 37.7 5,536 3,388 61.2 
160 922 248 26.9 1,771 391 22.1 5,536 719 13.0 
168 922 405 43.9 1,771 652 36.8 5,536 1,196 21.6 
176 922 637 69.1 1,771 960 54.2 5,536 1,828 33.0 
3 192 922 899 97.5 1,771 1,613 91.1 5,536 3,291 59.4 
208 922 674 73.1 1,771 1,294 73.1 5,536 3,706 66.9 
216 922 463 50.2 1,771 926 52.3 5,536 3,233 58.4 
224 922 265 28.7 1,771 526 29.7 5,536 2,370 42.8 
170 1,004 298 29.7 1,910 578 30.3 5,536 1,344 24.3 
177 1,004 467 46.5 1,910 907 47.5 5,536 1,892 34.2 

185 1,004 738 73.5 1,910 1,336 69.9 5,536 2,717 49.1 
4 201 1,004 979 97.5 1,910 1,705 89.3 5,536 3,697 66.8 
216 1,004 736 73.3 1,910 1,150 60.2 5,536 3,246 58.6 
224 1,004 432 43.0 1,910 686 35.9 5,536 2,383 43.0 
231 1,004 238 23.7 1,910 415 21.7 5,536 1,657 29.9 
177 681 258 37.9 1,730 575 33.2 5,536 1,904 34.4 
184 681 420 61.7 1,730 900 52.0 5,536 2,584 46.7 
192 681 594 87.2 1,730 1,322 76.4 5,536 3,266 59.0 
5 207 681 680 99.9 1,730 1,598 92.4 5,536 3,729 67.4 
222 681 478 70.2 1,730 980 56.6 5,536 2,609 47.1 
229 681 297 43.6 1,730 642 37.1 5,536 1,855 33.5 
236 681 189 27.8 1,730 384 22.2 5,536 1,196 21.6 
181 512 130 25.4 1,284 423 32.9 5,536 2,309 41.7 
189 512 263 51.4 1,284 675 52.6 5,536 2,988 54.0 
196 512 406 79.3 1,284 958 74.6 5,536 3,558 64.3 
6 211 512 509 99.4 1,284 1,203 93.7 5,536 3,636 65.7 
226 512 448 87.5 1,284 846 65.9 5,536 2,173 39.3 
233 512 307 60.0 1,284 571 44.5 5,536 1,486 26.8 
241 512 179 35.0 1,284 338 26.3 5,536 858 15.5 
185 509 133 26.1 1,005 246 24.5 5,536 2,692 48.6 

193 509 267 52.5 1,005 473 47.1 5,536 3,384 61.1 
200 509 374 73.5 1,005 718 71.4 5,536 3,676 66.4 
7 215 509 504 99.0 1,005 978 97.3 5,536 3,362 60.7 
230 509 447 87.8 1,005 785 78.1 5,536 1,727 31.2 
237 509 332 65.2 1,005 533 53.0 5,536 1,125 20.3 
244 509 205 40.3 1,005 335 33.3 5,536 733 13.2 
189 366 120 32.8 861 227 26.4 5,536 3,035 54.8 
196 366 209 57.1 861 435 50.5 5,536 3,527 63.7 
204 366 306 83.6 861 660 76.7 5,536 3,702 66.9 
8 218 366 363 99.2 861 852 99.0 5,536 3,011 54.4 
233 366 342 93.4 861 692 80.4 5,536 1,479 26.7 
241 366 248 67.8 861 460 53.4 5,536 861 15.6 

248 366 159 43.4 861 296 34.4 5,536 509 9.2 
Min. - - 22.0 - - 12.4 - - 4.2 
Max. - - 99.9 - - 99.0 - - 67.4 
Mean - - 60.8 - - 53.9 - - 42.3 
SD - - 25.2 - - 23.9 - - 19.6 
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Table D.4. Pool Utilization for Off-Grade Simulees—Mathematics 


Appendix D: Pool Utilization 


On-Grade Item Pool +1Grade Item Pool All-Grade Item Pool 

Total Items Used Total Items Used Total Items Used 

Grade True RIT #iltems N % #iltems N % #items N % 
133 689 88 12.8 1,926 124 6.4 4,793 180 3.8 
138 689 126 18.3 1,926 167 8.7 4,793 275 5.7 
9 142 689 179 26.0 1,926 232 12.0 4,793 384 8.0 
216 689 171 24.8 1,926 719 37.3 4,793 2,581 53.8 
220 689 136 19.7 1,926 579 30.1 4,793 2,445 51.0 
224 689 109 15.8 1,926 447 23.2 4,793 2,319 48.4 
151 928 96 10.3 2,543 355 14.0 4,793 663 13.8 
159 928 183 19.7 2,543 573 22.5 4,793 997 20.8 
3 162 928 232 25.0 2,543 685 26.9 4,793 1,143 23.8 
228 928 321 34.6 2,543 788 31.0 4,793 2,063 43.0 
232 928 257 27.7 2,543 651 25.6 4,793 1,868 39.0 
236 928 215 23.2 2,543 497 19.5 4,793 1,525 31.8 
164 1,020 120 11.8 2,598 337 13.0 4,793 1,222 25.5 
168 1,020 156 15.3 2,598 437 16.8 4,793 1,410 29.4 
4 172 1,020 195 19.1 2,598 574 22.1 4,793 1,614 33.7 
239 1,020 323 31.7 2,598 826 31.8 4,793 1,343 28.0 
243 1,020 252 24.7 2,598 650 25.0 4,793 1,089 22.7 
247 1,020 180 17.6 2,598 497 19.1 4,793 862 18.0 
174 822 116 14.1 2,815 341 12.1 4,793 1,668 34.8 
176 822 118 14.4 2,815 374 13.3 4,793 1,757 36.7 
5 179 822 142 17.3 2,815 450 16.0 4,793 1,895 39.5 
248 822 380 46.2 2,815 852 30.3 4,793 805 16.8 
249 822 361 43.9 2,815 791 28.1 4,793 730 15.2 
251 822 330 40.1 2,815 718 25.5 4,793 623 13.0 
180 1,166 167 14.3 2,742 300 10.9 6,657 829 12.5 
181 1,166 177 15.2 2,742 293 10.7 6,657 893 13.4 

6 182 1,166 209 17.9 2,742 314 11.5 6,657 941 14.1 
254 1,166 434 37.2 2,742 887 32.3 6,657 2,656 39.9 
257 1,166 379 32.5 2,742 746 27.2 6,657 2,362 35.5 
260 1,166 300 25.7 2,742 607 22.1 6,657 2,106 31.6 
183 945 120 12.7 2,716 250 9.2 6,657 1,019 15.3 
184 945 123 13.0 2,716 275 10.1 6,657 1,112 16.7 
7 185 945 125 13.2 2,716 296 10.9 6,657 1,126 16.9 
262 945 322 34.1 2,716 821 30.2 6,657 1,949 29.3 
264 945 310 32.8 2,716 696 25.6 6,657 1,767 26.5 

267 945 255 27.0 2,716 599 22.1 6,657 1,538 23.1 
187 718 64 8.9 2,824 173 6.1 6,657 1,199 18.0 
189 718 73 10.2 2,824 178 6.3 6,657 1,348 20.2 
8 190 718 71 9.9 2,824 210 74 6,657 1,442 21.7 
268 718 331 46.1 2,824 1,316 46.6 6,657 1,448 21.8 
269 718 313 43.6 2,824 1,208 42.8 6,657 1,357 20.4 
270 718 300 41.8 2,824 1,183 41.9 6,657 1,290 19.4 
Min. - - 8.9 - - 6.1 - - 3.8 
Max. - - 46.2 - - 46.6 - - 53.8 

Mean - - 23.6 - - 21.1 - - 25.1 

SD - - 11.2 - - 10.7 - - 12.1 
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Table D.5. Pool Utilization for Off-Grade Simulees—Reading 


Appendix D: Pool Utilization 


On-Grade Item Pool +1Grade Item Pool All-Grade Item Pool 

Total Items Used Total Items Used Total Items Used 

Grade True RIT #iltems N % #iltems N % #iltems N % 
134 584 63 10.8 1,324 71 5.4 5,727 76 1.3 
138 584 75 12.8 1,324 80 6.0 5,727 112 2.0 
9 142 584 88 15.1 1,324 118 8.9 5,727 160 2.8 
218 584 78 13.4 1,324 300 22.7 5,727 2,515 43.9 
223 584 58 9.9 1,324 194 14.7 5,727 1,929 33.7 

227 584 53 9.1 1,324 128 9.7 5,727 1,435 25.1 
145 1,072 92 8.6 2,012 123 6.1 5,727 218 3.8 

150 1,072 124 11.6 2,012 217 10.8 5,727 351 6.1 
3 154 1,072 189 17.6 2,012 322 16.0 5,727 543 9.5 
229 1,072 100 9.3 2,012 218 10.8 5,727 1,241 21.7 
232 1,072 83 7.7 2,012 165 8.2 5,727 969 16.9 
234 1,072 65 6.1 2,012 135 6.7 5,727 768 13.4 
157 1,216 127 10.4 2,271 285 12.5 5,727 722 12.6 
161 1,216 196 16.1 2,271 402 17.7 5,727 962 16.8 
4 165 1,216 270 22.2 2,271 561 24.7 5,727 1,219 21.3 
236 1,216 111 9.1 2,271 184 8.1 5,727 677 11.8 
238 1,216 84 6.9 2,271 154 6.8 5,727 539 9.4 
241 1,216 66 5.4 2,271 104 4.6 5,727 366 6.4 
167 969 139 14.3 2,441 325 13.3 5,727 1,348 23.5 
170 969 179 18.5 2,441 417 17.1 5,727 1,585 27.7 
5 172 969 202 20.8 2,441 464 19.0 5,727 1,708 29.8 
242 969 88 9.1 2,441 233 9.5 5,727 335 5.8 

244 969 67 6.9 2,441 188 7.7 5,727 276 4.8 

245 969 61 6.3 2,441 164 6.7 5,727 272 4.7 
174 1,036 112 10.8 2,106 289 13.7 6,472 941 14.5 
176 1,036 137 13.2 2,106 319 15.1 6,472 1,099 17.0 
6 179 1,036 164 15.8 2,106 433 20.6 6,472 1,402 21.7 
246 1,036 130 12.5 2,106 228 10.8 6,472 870 13.4 
248 1,036 110 10.6 2,106 197 9.4 6,472 700 10.8 
249 1,036 102 9.8 2,106 168 8.0 6,472 688 10.6 

180 697 115 16.5 1,893 223 11.8 6,472 1,495 23.1 

182 697 132 18.9 1,893 288 15.2 6,472 1,687 26.1 
7 183 697 135 19.4 1,893 314 16.6 6,472 1,844 28.5 
250 697 118 16.9 1,893 228 12.0 6,472 559 8.6 

252 697 95 13.6 1,893 194 10.2 6,472 487 75 

253 697 95 13.6 1,893 165 8.7 6,472 441 6.8 
184 710 102 14.4 1,382 181 13.1 6,472 1,906 29.4 

186 710 115 16.2 1,382 207 15.0 6,472 2,143 33.1 
8 187 710 124 17.5 1,382 247 17.9 6,472 2,257 34.9 
254 710 130 18.3 1,382 196 14.2 6,472 392 6.1 

255 710 122 17.2 1,382 166 12.0 6,472 369 5.7 

256 710 118 16.6 1,382 151 10.9 6,472 297 4.6 
Min. - - 5.4 - - 4.6 - - ilps) 
Max. - - 22.2 - - 24.7 - - 43.9 
Mean - - 13.1 - - 12.1 - - 15.7 
SD - - 4.4 - - 4.8 - - 10.8 
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Table D.6. Pool Utilization for Off-Grade Simulees—Language Usage 


Appendix D: Pool Utilization 


On-Grade Item Pool +1Grade Item Pool All-Grade Item Pool 

Total Items Used Total Items Used Total Items Used 

Grade True RIT #iltems N % #iltems N % #iltems N % 
134 445 58 13.0 1,621 64 3.9 5,536 75 1.4 
138 445 60 13.5 1,621 93 5.7 5,536 100 1.8 
9 142 445 71 16.0 1,621 129 8.0 5,536 151 2.7 
216 445 153 34.4 1,621 531 32.8 5,536 3,220 58.2 
220 445 125 28.1 1,621 398 24.6 5,536 2,794 50.5 
224 445 79 17.8 1,621 271 16.7 5,536 2,383 43.0 
155 922 184 20.0 1,771 262 14.8 5,536 495 8.9 

157 922 203 22.0 1,771 312 17.6 5,536 616 11.1 
3 159 922 242 26.2 1,771 387 21.9 5,536 666 12.0 
226 922 214 23.2 1,771 445 25.1 5,536 2,184 39.5 
228 922 194 21.0 1,771 370 20.9 5,536 1,962 35.4 
231 922 128 13.9 1,771 317 17.9 5,536 1,696 30.6 
160 1,004 134 13.3 1,910 303 15.9 5,536 719 13.0 

165 1,004 201 20.0 1,910 415 21.7 5,536 1,003 18.1 

4 169 1,004 268 26.7 1,910 550 28.8 5,536 1,278 23.1 
232 1,004 223 22.2 1,910 369 19.3 5,536 1,612 29.1 
234 1,004 183 18.2 1,910 313 16.4 5,536 1,431 25.8 
236 1,004 141 14.0 1,910 261 13.7 5,536 1,217 22.0 
170 681 122 17.9 1,730 347 20.1 5,536 1,344 24.3 
173 681 176 25.8 1,730 423 24.5 5,536 1,564 28.3 
5 176 681 250 36.7 1,730 510 29.5 5,536 1,828 33.0 
237 681 139 20.4 1,730 341 19.7 5,536 1,114 20.1 
239 681 127 18.6 1,730 305 17.6 5,536 994 18.0 
241 681 90 13.2 1,730 223 12.9 5,536 861 15.6 
177 512 99 19.3 1,284 294 22.9 5,536 1,904 34.4 
179 512 122 23.8 1,284 386 30.1 5,536 2,094 37.8 

6 180 512 124 24.2 1,284 391 30.5 5,536 2,219 40.1 
242 512 169 33.0 1,284 319 24.8 5,536 784 14.2 
243 512 140 27.3 1,284 295 23.0 5,536 744 13.4 
244 512 117 22.9 1,284 274 21.3 5,536 693 12.5 
181 509 102 20.0 1,005 179 17.8 5,536 2,309 41.7 
183 509 125 24.6 1,005 209 20.8 5,536 2,491 45.0 
7 184 509 122 24.0 1,005 242 24.1 5,536 2,573 46.5 
245 509 198 38.9 1,005 315 31.3 5,536 650 11.7 

247 509 170 33.4 1,005 302 30.0 5,536 558 10.1 
248 509 158 31.0 1,005 251 25.0 5,536 513 9.3 
185 366 82 22.4 861 172 20.0 5,536 2,692 48.6 
186 366 89 24.3 861 195 22.6 5,536 2,732 49.3 
8 188 366 118 32.2 861 217 25.2 5,536 3,017 54.5 
249 366 141 38.5 861 258 30.0 5,536 457 8.3 
250 366 132 36.1 861 255 29.6 5,536 424 7.7 
251 366 119 32.5 861 235 27.3 5,536 421 7.6 
Min. - - 13.0 - - 3.9 - - 1.4 
Max. - - 38.9 - - 32.8 - - 58.2 
Mean - - 23.9 - - 21.6 - - 25.2 

SD - - 7.4 - - 6.8 - - 16.1 
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Appendix E: Content Balancing 


Appendix E: Content Balancing 


Table E.1. Content Balancing for On-Grade Simulees—Mathematics 


On-Grade Item Pool +1 Grade Item Pool All-Grade Item Pool 
True Slightly Extremely Slightly Extremely Slightly Extremely 
Grade} RIT Balanced Unbalanced Unbalanced | Balanced Unbalanced Unbalanced | Balanced Unbalanced Unbalanced 
151 100.0 - - 100.0 - - 100.0 - - 
159 100.0 - - 100.0 - - 100.0 - - 
167 99.8 0.2 - 100.0 - - 100.0 - - 
2 183 100.0 - - 100.0 - - 100.0 - - 
199 99.8 0.2 - 100.0 - - 100.0 - - 
207 100.0 - - 99.8 0.2 - 100.0 - - 
215 100.0 - - 99.4 0.6 - 100.0 - - 
164 99.8 0.2 - 100.0 - - 100.0 - - 
172 100.0 - - 100.0 - - 100.0 - - 
179 100.0 - - 100.0 - - 99.9 0.1 - 
3 194 100.0 - - 100.0 - - 100.0 - - 
209 100.0 - - 100.0 - - 100.0 - - 
216 99.4 0.6 - 100.0 - - 99.9 0.1 - 
224 99.1 0.9 - 100.0 - - 99.7 0.2 0.1 
174 99.8 0.2 - 100.0 - - 99.9 0.1 - 
181 98.7 1.3 - 100.0 - - 100.0 - - 
189 98.9 1.1 - 100.0 - - 100.0 - - 
4 205 99.9 0.1 - 99.9 0.1 - 100.0 - - 
220 100.0 - - 99.2 0.8 - 99.6 0.4 - 
228 100.0 - - 97.1 2.9 - 99.7 0.3 - 
236 99.9 0.1 - 97.6 2.4 - 99.9 0.1 - 
180 97.6 2.5 - 80.8 1.1 18.2 100.0 - - 
188 94.4 5.6 - 80.4 2.5 17.1 100.0 - - 
197 93.8 6.2 - 80.2 4.3 15.5 100.0 - - 
5 214 88.0 11.4 0.6 82.5 9.3 8.2 100.0 - - 
230 80.1 18.9 1.0 93.0 6.0 1.0 99.7 0.3 - 
239 87.1 12.4 0.5 92.9 7.1 - 100.0 - - 
247 94.6 5.3 0.1 86.5 13.5 - 100.0 - - 
183 95.1 4.9 - 96.2 3.7 0.1 98.5 1.5 - 
192 91.0 8.9 0.1 92.2 6.8 1.0 96.5 3.5 - 
200 85.7 13.5 0.8 88.9 9.4 1.7 94.9 5.1 - 
6 218 82.0 16.7 1.3 85.4 10.1 4.5 97.4 2.6 - 
234 97.2 2.8 - 81.3 8.5 10.2 100.0 - - 
243 98.9 1.2 - 80.7 3.2 16.1 100.0 - - 
251 99.4 0.7 - 81.3 1.1 17.5 99.8 0.2 - 
187 97.2 2.8 - 96.0 3.8 0.2 97.6 2.4 - 
196 98.9 1.1 - 97.0 2.9 0.1 95.4 4.5 0.1 
205 99.6 0.4 - 96.1 3.8 0.1 95.8 4.1 0.1 
7 223 99.9 0.1 - 99.5 0.5 - 99.2 0.8 - 
242 100.0 - - 100.0 - - 100.0 - - 
251 100.0 - - 100.0 - - 99.8 0.2 - 
260 100.0 - - 100.0 - - 99.8 0.2 - 
190 100.0 - - 99.9 0.1 - 96.4 3.6 - 
200 100.0 - - 99.9 0.1 - 95.3 4.7 - 
209 100.0 - - 99.9 0.1 - 96.2 3.8 - 
8 229 92.4 7.4 0.2 100.0 - - 99.9 0.1 - 
248 92.5 7.2 0.3 99.5 0.5 - 99.8 0.2 - 
258 96.9 3.1 - 99.6 0.4 - 99.8 0.2 - 
267 98.6 1.5 - 99.7 0.3 - 99.8 0.2 - 
Min. 80.1 0.1 0.1 80.2 0.1 0.1 94.9 0.1 0.1 
Max. 100.0 18.9 1.3 100.0 13.5 18.2 100.0 tora 0.1 
Mean 97.0 4.4 0.5 95.6 6h) 7.4 99.2 des) 0.1 
SD 4.9 By 0.4 6.9 3.7 7.5 1.5 1.8 0.0 
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Appendix E: Content Balancing 


Table E.2. Content Balancing for On-Grade Simulees—Reading 


On-Grade Item Pool +1 Grade Item Pool All-Grade Item Pool 
True Slightly Extremely Slightly Extremely Slightly Extremely 
Grade} RIT Balanced Unbalanced Unbalanced | Balanced Unbalanced Unbalanced | Balanced Unbalanced Unbalanced 
145 100.0 - - 99.9 - 0.1 100.0 - - 
154 99.7 0.2 0.1 99.9 - 0.1 99.8 - 0.2 
163 99.1 0.7 0.2 99.8 0.1 0.1 99.8 - 0.1 
2 180 99.8 - - 100.0 - - 100.0 - - 
198 100.0 - - 99.9 - 0.1 100.0 - - 
207 100.0 - - 100.0 - - 100.0 - - 
216 100.0 - - 100.0 - - 99.9 - 0.1 
157 100.0 - - 99.6 0.4 - 99.9 0.1 - 
165 99.9 0.1 - 99.2 0.7 0.1 99.7 0.2 0.1 
174 99.9 0.1 - 99.9 0.1 - 99.8 - 0.2 
3 192 99.9 0.1 - 99.9 0.1 - 100.0 - - 
209 100.0 - - 100.0 - - 100.0 - - 
218 100.0 - - 100.0 - - 100.0 - - 
227 100.0 - - 100.0 - - 99.9 0.1 - 
167 97.6 2.0 0.4 99.2 0.8 - 99.6 0.2 0.2 
175 99.9 0.1 - 99.6 0.2 0.2 100.0 - - 
184 100.0 - - 99.9 0.1 - 100.0 - - 
4 201 99.9 0.1 - 100.0 - - 100.0 - - 
218 100.0 - - 100.0 - - 100.0 - - 
226 100.0 - - 100.0 - - 100.0 - - 
234 100.0 - - 100.0 - - 99.4 0.6 - 
174 100.0 - - 99.6 0.4 - 99.7 - 0.3 
183 99.9 0.1 - 99.8 0.2 - 99.9 - 0.1 
181 99.9 0.1 - 100.0 - - 100.0 - - 
5 208 100.0 - - 100.0 - - 100.0 - - 
224 100.0 - - 100.0 - - 99.9 0.1 - 
232 100.0 - - 100.0 - - 99.7 0.3 - 
241 100.0 - - 100.0 - - 99.1 0.8 0.1 
180 81.3 8.0 10.7 99.8 0.2 - 100.0 - - 
188 89.2 6.8 4.0 99.7 0.3 - 100.0 - - 
196 95.8 3.0 1.2 99.8 0.2 - 100.0 - - 
6 212 100.0 - - 100.0 - - 100.0 - - 
229 100.0 - - 100.0 - - 100.0 - - 
237 100.0 - - 100.0 - - 100.0 - - 
245 100.0 - - 100.0 - - 100.0 - - 
184 82.5 12.2 5.3 89.6 8.0 2.4 100.0 - - 
192 87.8 8.2 4.0 91.6 5.7 2.7 100.0 - - 
200 93.8 4.2 2.0 97.3 1.9 0.8 99.9 - 0.1 
7 216 100.0 - - 100.0 - - 100.0 - - 
233 100.0 - - 100.0 - - 100.0 - - 
241 100.0 - - 100.0 - - 100.0 - - 
249 100.0 - - 100.0 - - 100.0 - - 
188 63.0 21.4 15.6 90.4 7.2 2.4 100.0 - - 
196 81.8 12.0 6.2 92.1 5.4 2.5 99.8 - 0.2 
204 95.4 3.1 1.5 98.3 0.9 0.8 99.8 - 0.2 
8 220 100.0 - - 100.0 - - 100.0 - - 
237 100.0 - - 100.0 - - 100.0 - - 
245 100.0 - - 100.0 - - 100.0 - - 
253 100.0 - - 100.0 - - 100.0 - - 
Min. 63.0 0.1 0.1 89.6 0.1 0.1 99.1 0.1 0.1 
Max. 100.0 21.4 15.6 100.0 8.0 2.7 100.0 0.8 0.3 
Mean 97.3 4.3 4.3 99.1 1.7 1.0 99.9 0.3 0.2 
SD 7.0 5.8 4.7 2.5 2.7 1.1 0.2 0.3 0.1 
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Table E.3. Content Balancing for On-Grade Simulees—Language Usage 


On-Grade Item Pool 


+1 Grade Item Pool 


All-Grade Item Pool 


True Slightly Extremely Slightly Extremely Slightly Extremely 
Grade} RIT Balanced Unbalanced Unbalanced | Balanced Unbalanced Unbalanced | Balanced Unbalanced Unbalanced 
147 100.0 - - 100.0 - - 100.0 - - 
155 100.0 - - 100.0 - - 100.0 - - 
164 100.0 - - 100.0 - - 100.0 - - 
2 180 100.0 - - 100.0 - - 100.0 - - 
197 100.0 - - 100.0 - - 100.0 - - 
206 100.0 - - 100.0 - - 100.0 - - 
214 100.0 - - 100.0 - - 100.0 - - 
160 100.0 - - 100.0 - - 100.0 - - 
168 100.0 - - 100.0 - - 100.0 - - 
176 100.0 - - 100.0 - - 100.0 - - 
3 192 100.0 - - 100.0 - - 100.0 - - 
208 100.0 - - 100.0 - - 100.0 - - 
216 100.0 - - 100.0 - - 100.0 - - 
224 100.0 - - 100.0 - - 100.0 - - 
170 100.0 - - 100.0 - - 100.0 - - 
177 100.0 - - 100.0 - - 100.0 - - 
185 100.0 - - 100.0 - - 100.0 - - 
4 201 100.0 - - 100.0 - - 100.0 - - 
216 100.0 - - 100.0 - - 100.0 - - 
224 100.0 - - 100.0 - - 100.0 - - 
231 100.0 - - 100.0 - - 100.0 - - 
177 100.0 - - 100.0 - - 100.0 - - 
184 100.0 - - 100.0 - - 100.0 - - 
192 100.0 - - 100.0 - - 100.0 - - 
5 207 100.0 - - 100.0 - - 100.0 - - 
222 100.0 - - 100.0 - - 100.0 - - 
229 100.0 - - 100.0 - - 100.0 - - 
236 100.0 - - 100.0 - - 100.0 - - 
181 100.0 - - 100.0 - - 100.0 - - 
189 100.0 - - 100.0 - - 100.0 - - 
196 100.0 - - 100.0 - - 100.0 - - 
6 211 100.0 - - 100.0 - - 100.0 - - 
226 100.0 - - 100.0 - - 100.0 - - 
233 100.0 - - 100.0 - - 100.0 - - 
241 100.0 - - 100.0 - - 100.0 - - 
185 100.0 - - 100.0 - - 100.0 - - 
193 100.0 - - 100.0 - - 100.0 - - 
200 100.0 - - 100.0 - - 100.0 - - 
7 215 100.0 - - 100.0 - - 100.0 - - 
230 100.0 - - 100.0 - - 100.0 - - 
237 100.0 - - 100.0 - - 100.0 - - 
244 100.0 - - 100.0 - - 100.0 - - 
189 100.0 - - 100.0 - - 100.0 - - 
196 100.0 - - 100.0 - - 100.0 - - 
204 100.0 - - 100.0 - - 100.0 - - 
8 218 100.0 - - 100.0 - - 100.0 - - 
233 100.0 - - 100.0 - - 100.0 - - 
241 100.0 - - 100.0 - - 100.0 - - 
248 100.0 - - 100.0 - - 100.0 - - 
Min. 100.0 - - 100.0 - - 100.0 - - 
Max. 100.0 - - 100.0 - - 100.0 - - 
Mean 100.0 - - 100.0 - - 100.0 - - 
SD 0.0 - - 0.0 - - 0.0 - - 
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Table E.4. Content Balancing for Off-Grade Simulees—Mathematics 


On-Grade Item Pool 


+1 Grade Item Pool 


All-Grade Item Pool 


True Slightly Extremely Slightly Extremely Slightly Extremely 
Grade} RIT Balanced Unbalanced Unbalanced | Balanced Unbalanced Unbalanced | Balanced Unbalanced Unbalanced 
133 100.0 - - 100.0 - - 100.0 - - 
138 100.0 - - 100.0 - - 100.0 - - 
9 142 100.0 - - 100.0 - - 100.0 - - 
216 100.0 - - 99.4 0.6 - 99.9 0.1 - 
220 100.0 - - 99.8 0.1 0.1 99.6 0.4 - 
224 100.0 - - 99.5 0.5 - 99.7 0.3 - 
151 99.9 0.1 - 100.0 - - 100.0 - - 
159 99.3 0.7 - 100.0 - - 100.0 - - 
3 162 99.8 0.2 - 100.0 - - 100.0 - - 
228 98.9 41 - 99.9 0.1 - 99.5 0.5 - 
232 96.7 3.2 - 99.4 0.5 0.1 99.9 0.1 - 
236 95.4 4.6 - 98.6 1.3 0.1 99.6 0.4 - 
164 100.0 - - 99.7 0.3 - 100.0 - - 
168 100.0 - - 100.0 - - 100.0 - - 
4 172 99.9 - - 100.0 - - 100.0 - - 
239 99.8 0.2 - 97.5 2.5 - 99.9 0.1 - 
243 99.3 0.7 - 99.0 1.0 - 100.0 - - 
247 99.5 0.5 - 99.7 0.3 - 100.0 - - 
174 99.6 0.4 - 80.8 0.3 18.9 99.9 0.1 - 
176 99.3 0.4 0.3 80.8 0.9 18.3 100.0 - - 
5 179 99.3 0.7 - 80.4 0.9 18.7 99.9 0.1 - 
248 95.4 4.6 - 86.7 13.3 - 100.0 - - 
249 96.9 3.1 - 85.4 14.6 - 100.0 - - 
251 98.3 1.8 - 83.7 16.3 - 99.9 0.1 - 
180 96.2 3.8 - 97.2 2.7 0.1 99.4 0.6 - 
181 95.5 4.5 - 97.4 2.5 0.1 98.8 1.2 - 
6 182 95.0 4.9 0.1 96.7 3.1 0.2 98.8 1:2 - 
254 99.9 0.1 - 82.0 0.5 17.5 99.5 0.5 - 
257 100.0 - - 82.7 0.2 17.1 99.9 0.1 - 
260 100.0 - - 84.0 0.2 15.8 99.7 0.4 - 
183 98.1 1.9 - 98.1 1.9 - 98.5 1.5 - 
184 98.6 1.4 - 97.6 2.3 0.1 98.0 2.0 - 
7 185 97.7 2.3 - 97.0 2.9 0.1 97.5 2.5 - 
262 99.9 0.1 - 100.0 - - 99.9 0.1 - 
264 100.0 - - 100.0 - - 99.7 0.3 - 
267 100.0 - - 100.0 - - 99.6 0.4 - 
187 100.0 - - 99.8 0.2 - 97.6 2.4 - 
189 100.0 - - 99.8 0.2 - 96.8 3.2 - 
8 190 100.0 - - 99.9 0.1 - 97.2 2.8 - 
268 98.3 1.7 - 99.4 0.5 0.1 99.6 0.4 - 
269 98.0 2.0 - 99.9 0.1 - 99.6 0.4 - 
270 98.3 1.7 - 99.5 0.5 - 99.6 0.4 - 
Min. 95.0 0.1 0.1 80.4 0.1 0.1 96.8 0.1 - 
Max. 100.0 4.9 0.3 100.0 16.3 18.9 100.0 S22 - 
Mean 98.9 1.8 0.2 95.7 223) 7.2 99.4 0.8 - 
SD 1.5 1.6 0.1 6.9 4.3 9.0 0.8 0.9 - 
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Table E.5. Content Balancing for Off-Grade Simulees—Reading 


On-Grade Item Pool +1 Grade Item Pool All-Grade Item Pool 
True Slightly Extremely Slightly Extremely Slightly Extremely 
Grade} RIT Balanced Unbalanced Unbalanced | Balanced Unbalanced Unbalanced | Balanced Unbalanced Unbalanced 
134 19.9 80.1 - 100.0 - - 100.0 - - 
138 19.9 80.1 - 100.0 - - 100.0 - - 
9 142 19.9 80.1 - 100.0 - - 100.0 - - 
218 18.2 81.8 - 100.0 - - 100.0 - - 
223 19.0 81.0 - 100.0 - - 100.0 - - 
227 19.7 80.3 - 100.0 - - 99.9 0.1 - 
145 100.0 - - 100.0 - - 100.0 - - 
150 100.0 - - 99.9 0.1 - 99.9 0.1 - 
3 154 100.0 - - 99.7 0.3 - 99.8 0.2 - 
229 100.0 - - 100.0 - - 99.9 0.1 - 
232 100.0 - - 100.0 - - 99.7 0.3 - 
234 100.0 - - 100.0 - - 99.2 0.8 - 
157 90.2 8.4 1.4 99.2 0.8 - 99.9 0.1 - 
161 92.6 6.1 1.3 99.1 0.8 0.1 99.6 0.4 - 
4 165 96.3 3.3 0.4 99.1 0.6 0.3 99.6 0.2 0.2 
236 100.0 - - 100.0 - - 99.8 0.2 - 
238 100.0 - - 100.0 - - 98.9 0.9 0.2 
241 100.0 - - 100.0 - - 99.2 0.8 - 
167 100.0 - - 98.9 1.0 0.1 99.6 0.4 - 
170 100.0 - - 99.0 0.8 0.2 99.8 0.2 - 
5 172 100.0 - - 99.4 0.6 - 99.8 0.2 - 
242 100.0 - - 100.0 - - 98.9 1.0 0.1 
244 100.0 - - 100.0 - - 99.2 0.7 0.1 
245 100.0 - - 100.0 - - 99.3 0.6 0.1 
174 79.7 7.1 13.2 100.0 - - 100.0 - - 
176 80.2 7.6 12.2 99.8 0.2 - 100.0 - - 
6 179 82.3 7.1 10.6 100.0 - - 100.0 - - 
246 100.0 - - 100.0 - - 100.0 - - 
248 100.0 - - 100.0 - - 100.0 - - 
249 100.0 - - 100.0 - - 100.0 - - 
180 80.5 12.9 6.6 90.0 8.8 1.2 100.0 - - 
182 81.7 12.6 5.7 88.9 9.1 2.0 100.0 - - 
7 183 81.6 12.4 6.0 89.6 8.5 1.9 100.0 - - 
250 100.0 - - 100.0 - - 100.0 - - 
252 100.0 - - 100.0 - - 100.0 - - 
253 100.0 - - 100.0 - - 100.0 - - 
184 58.9 23.0 18.1 88.5 9.5 2.0 100.0 - - 
186 58.3 25.1 16.6 89.1 8.9 2.0 100.0 - - 
8 187 58.4 25.1 16.5 88.9 9.1 2.0 100.0 - - 
254 100.0 - - 100.0 - - 100.0 - - 
255 100.0 - - 100.0 - - 100.0 - - 
256 100.0 - - 100.0 - - 100.0 - - 
Min. 18.2 3.3 0.4 88.5 0.1 0.1 98.9 0.1 0.1 
Max. 100.0 81.8 18.1 100.0 9.5 2.0 100.0 1.0 0.2 
Mean 82.3 35.2 9.1 98.3 3.9 1.2 99.8 0.4 0.1 
SD 28.5 33.6 6.4 3.8 4.3 0.9 0.3 0.3 0.1 
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Table E.6. Content Balancing for Off-Grade Simulees—Language Usage 


On-Grade Item Pool 


+1 Grade Item Pool 


All-Grade Item Pool 


True Slightly Extremely Slightly Extremely Slightly Extremely 
Grade | RIT Balanced Unbalanced Unbalanced | Balanced Unbalanced Unbalanced | Balanced Unbalanced Unbalanced 
134 100.0 - - 100.0 - - 100.0 - - 
138 100.0 - - 100.0 - - 100.0 - - 
9 142 100.0 - - 100.0 - - 100.0 - - 
216 100.0 - - 100.0 - - 100.0 - - 
220 100.0 - - 100.0 - - 100.0 - - 
224 100.0 - - 100.0 - - 100.0 - - 
155 100.0 - - 100.0 - - 100.0 - - 
157 100.0 - - 100.0 - - 100.0 - - 
3 159 100.0 - - 100.0 - - 100.0 - - 
226 100.0 - - 100.0 - - 100.0 - - 
228 100.0 - - 100.0 - - 100.0 - - 
231 100.0 - - 100.0 - - 100.0 - - 
160 100.0 - - 100.0 - - 100.0 - - 
165 100.0 - - 100.0 - - 100.0 - - 
4 169 100.0 - - 100.0 - - 100.0 - - 
232 100.0 - - 100.0 - - 100.0 - - 
234 100.0 - - 100.0 - - 100.0 - - 
236 100.0 - - 100.0 - - 100.0 - - 
170 100.0 - - 100.0 - - 100.0 - - 
173 100.0 - - 100.0 - - 100.0 - - 
5 176 100.0 - - 100.0 - - 100.0 - - 
237 100.0 - - 100.0 - - 100.0 - - 
239 100.0 - - 100.0 - - 100.0 - - 
241 100.0 - - 100.0 - - 100.0 - - 
177 100.0 - - 100.0 - - 100.0 - - 
179 100.0 - - 100.0 - - 100.0 - - 
6 180 100.0 - - 100.0 - - 100.0 - - 
242 100.0 - - 100.0 - - 100.0 - - 
243 100.0 - - 100.0 - - 100.0 - - 
244 100.0 - - 100.0 - - 100.0 - - 
181 100.0 - - 100.0 - - 100.0 - - 
183 100.0 - - 100.0 - - 100.0 - - 
7 184 100.0 - - 100.0 - - 100.0 - - 
245 100.0 - - 100.0 - - 100.0 - - 
247 100.0 - - 100.0 - - 100.0 - - 
248 100.0 - - 100.0 - - 100.0 - - 
185 100.0 - - 100.0 - - 100.0 - - 
186 100.0 - - 100.0 - - 100.0 - - 
8 188 100.0 - - 100.0 - - 100.0 - - 
249 100.0 - - 100.0 - - 100.0 - - 
250 100.0 - - 100.0 - - 100.0 - - 
251 100.0 - - 100.0 - - 100.0 - - 
Min. 100.0 - - 100.0 - - 100.0 - - 
Max. 100.0 - - 100.0 - - 100.0 - - 
Mean 100.0 - - 100.0 - - 100.0 - - 
SD 0.0 - - 0.0 - - 0.0 - - 
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Figure E.1. Item RIT Distribution by Instructional Area—Mathematics Grade 5 Item Pool 


= Geometry 

= Measurement and Data 

= Number and Operations 

= Operations and Algebraic Thinking 
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Figure E.2. Item RIT Distribution by Instructional Area—Mathematics Grade 6 Item Pool 


= Geometry 

= Operations and Algebraic Thinking 

= Statistics and Probability 

= The Real and Complex Number Systems 
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Figure E.3. Item RIT Distribution by Instructional Area—Mathematics Grade 8 Item Pool 


= Geometry 

= Operations and Algebraic Thinking 

= Statistics and Probability 

= The Real and Complex Number Systems 
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= Informational Text: Key Ideas and Details 
= Informational Text: Language, Craft, and Structure 
= Literary Text: Key Ideas and Details 


= Literary Text: Language, Craft, and Structure 
= Vocabulary: Acquisition and Use 
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Figure E.5. Item RIT Distribution by Instructional Area—Reading Grade 7 Item Pool 


= Informational Text: Key Ideas and Details 
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= Vocabulary: Acquisition and Use 


195 200 205 210 215 220 225 230 235 240 245 250 
RIT 


= Informational Text: Key Ideas and Details 
= Informational Text: Language, Craft, and Structure 
= Literary Text: Key Ideas and Details 


= Literary Text: Language, Craft, and Structure 
ie ia = Vocabulary: Acquisition and Use 
SSS WK. 


ie 


-_ 


le 


190 195 200 205 210 215 220 225 230 235 240 245 250 
RIT 


MAP® Growth™ Item Pool Simulation Study Page 43 


